How Software Can Hamper SSD Performance

My company has received some questions lately asking why we think SSDs will cause a lot of software to need reconfiguration.  This can be relatively easily answered with the text below, which is copied out of our most recent report: How Many IOPS is Enough?  This report was co-authored with Coughlin Associates.  (The full report can be purchased for immediate download on the Objective Analysis website.)  Here it is:

Why Software Latency is an Issue

Although SSDs will usually provide a significant boost to system performance, the full speed advantage of an SSD cannot be realized by a system as long as the device is still handled as if the NAND is just another storage volume. Software support for storage has never been written to perform quickly since software delays have historically been significantly shorter than the delays of the storage medium itself.

This is a problem that was explained in a very reasonable way by the University of California San Diego’s Computer Science and Engineering Department at the August 2009 Flash Memory Summit.

The graphic above shows us that the relative latency of a hard disk drive (left), an SSD (center), and on-board NVM (right) are orders of magnitude different from each other. Compared to the latency of the HDD and the SSD the fast NVM doesn’t even show on this chart.  Likewise, the File System overhead (red) and the time to perform a system call (black) are small enough compared to the HDD and SSD latency to be completely hidden in this chart.

Let’s look at the numbers that drive this chart. In the table below we provide the numbers that drive the chart above, all in microseconds (μs).

Breakdown of Latency in Different Media

Source: UCSD., Used with Permission

The hardware latency of the HDD overwhelms that of the file system overhead and the system call.  These two combine to less than 0.1% of the overall latency, so they have never demanded much focus from software developers.

With the SSD things change a bit, but not that significantly. Even though the SSD itself has 1/39th the latency of the HDD, the file system overhead and the speed of a system call still total less than 2.5% of the overall latency.

In the case of the fast NVM, however, things change significantly. The NVM’s 0.08μs latency is dwarfed in comparison with the file system overhead and the speed of a system call.  To make a very graphical point, we use the same data as the figure above and the table below, but this time we show it as percentages of the overall latency.

Clearly, the system software becomes the limiting factor in a system that is built using fast nonvolatile memory on the motherboard.

UCSD has written software to address this issue, which the university hopes to be able to license to interested parties.

Further Reading

The report that this was excerpted from: How Many IOPS is Enough? is based on a user survey that was designed to determine (among other important questions) what these users’ systems’ next bottleneck is once an SSD has cured the problem of storage speed.

Readers who are interested in a much deeper explanation of the inner workings of software that impedes SSD performance may want to view a SNIA webinar by Tom West and Eden Kim called: How Your Data Gets to Your SSD.