Every server listing quotes storage as a line item: 2x NVMe, 480 GB SSD, and so on. What the listing almost never tells you is which of the three numbers that actually matter for your workload. A video pipeline and a database can both be described as needing "fast storage", and they need opposite things. One wants to move gigabytes per second and does not care much about latency. The other moves tiny 8 KB pages all day and lives or dies on how long each one takes.
Here is what each metric measures, roughly what each class of disk delivers, and how to work out which one your application is bounded by before you commit to hardware.
IOPS is input/output operations per second: how many discrete read or write operations the device can complete each second. It matters when operations are small and frequent, which is the profile of databases, email servers, anything with an index, and container platforms writing overlay layers.
Throughput is bytes per second, quoted in MB/s or GB/s. It matters when operations are large and sequential: video rendering, backups, bulk imports, machine learning datasets streaming into memory. High throughput with mediocre IOPS describes a device that is excellent at large reads and poor at random ones.
Latency is how long a single operation takes, usually quoted as an average in milliseconds or microseconds. This is the number users actually feel. A page render that needs 40 database queries does not care how many IOPS the disk could theoretically do in parallel; it cares about 40 times the latency of each query's disk access. Latency is also the number most often degraded by neighbours on oversold storage, which is why a shared cloud volume and a dedicated NVMe can quote similar peak IOPS and feel nothing alike.
Rough figures for a single device under random load, the honest kind, not the vendor's best case with deep queues:
The important nuance: the jump from HDD to SATA SSD transformed most workloads. The jump from SATA SSD to NVMe transforms some of them, mainly databases and anything with heavy random write traffic, and barely registers for others, like serving static files or running a small web app that fits in the page cache. Paying for NVMe everywhere is paying for a number you will not use.
Measure before you buy. On an existing Linux box, iostat -x 5 gives you current IOPS, throughput and utilisation per device, and fio reproduces your application's access pattern properly. The two runs worth doing:
fio --name=randrw --rw=randrw --bs=4k --size=2g --numjobs=4 --runtime=60 --group_reporting for a database-like random small-block pattern.fio --name=seq --rw=read --bs=1m --size=8g --runtime=60 for a streaming or backup-like sequential pattern.Then compare against what the application demands. A few rules of thumb we use when sizing customers:
fsync traffic: this is where NVMe earns its price. Write latency per transaction is multiplied across every commit, and cutting 0.5 ms off each fsync can double sustainable transaction rate.Peak IOPS on shared storage are a ceiling, not a promise. Cloud volumes quote maximums you share with other tenants. If consistent latency matters, dedicated local disks are the only honest guarantee.
RAID changes the maths. Two NVMe in RAID 1 roughly double read IOPS and keep write latency low with a good controller or Linux mdadm. Four in RAID 10 can push a database further than one faster single disk. We cover the redundancy side of that decision in our dedicated versus cloud cost comparison.
Cache hides the disk until it does not. A server can benchmark beautifully from page cache and then collapse on a working set larger than RAM. Size fio tests above your RAM, and read the latency column, not just the IOPS column.
Our London node, and the platform generally, is built on all-flash NVMe because our own client services are the database-heavy kind of workload where latency under load is the difference between a fast site and a slow one at peak. The specifications are on the platform page. If you already have fio numbers for your workload and want a sanity check on sizing, mail admin@servercabin.com; out-of-band access and hardware management on these nodes are covered in our IPMI writeup.