Latency, IOPS & NVDIMMs

Latency vs IOPS Persectives - PowerPointSometimes it’s enlightening to compare several viewpoints on similar data.  At yesterday’s SNIA Persistent Memory Summit a number of presentations provided interesting overlapping views on certain subjects.

One of particular interest to The SSD Guy was latency vs. IOPS.  Tom Coughlin of Coughlin Associates and I presented the findings from our recently-published IOPS survey report and in Slide 19 displayed the basic chart behind this post’s graphic (click to enlarge, or, better yet, right-click to open in a new tab). This chart compares how many IOPS our respondents said they need for the storage in their most important application, and compared that to the latency they required from this storage.  For comparison’s sake we added a reference column on the left to roughly illustrate the latency of various standard forms of storage and memory.

You can see that we received a great variety of inputs spanning a very wide range of IOPS and latency needs, and that these didn’t all line up neatly as we would have anticipated.  One failing of this chart format is that it doesn’t account for multiple replies for the same IOPS/latency combination: If we had been able to include that the chart would have shown a clearer trendline running from the top left to the lower right.  Instead we have a band that broadly follows that trend of upper-left to lower-right.

Two other speakers presented the IOPS and latency that could be achieved through different system configurations.  This post’s graphic overlays their systems’ performance as yellow dots onto the survey chart so that they can be compared to the red diamonds that represent the survey responses.

The first speaker to provide such findings was Microsoft architect Tom Talpey.  Slide 15 of his presentation shows the performance of three systems: one based on a standard NAND NVMe SSD for storage, another using an NVDIMM-N in block transfer mode, and a third that coupled the newly-developed DAX system calls with the same NVDIMM-N.  DAX, an abbreviation for “Direct Access,” is a way for software to avoid managing storage in blocks, allowing data to be moved to and from the NVDIMM as easily and quickly as it can be moved into memory.  DAX has been added to recent releases of both Windows and Linux.

The graphic shows that the NVDIMM is naturally about an order of magnitude faster than the NVMe SSD when both are used in block mode, and when the NVDIMM is teamed with DAX the storage latency and IOPS improve by almost another order of magnitude.  This makes a powerful argument for using these two technologies together in fast storage applications.

In a later presentation Plexistor’s Chief Technology Officer Amit Golander shared the performance of his company’s storage system, one which combined NVMe SSDs with NVDIMM-Ns under the control of proprietary software.  The performance of that system, shown in Golander’s Slide 18, falls in between Talpey’s block mode and DAX NVDIMM benchmarks.

When we compare these systems’ performance to the latency and IOPS needs of our survey respondents we see that the four yellow dots neatly follow the rough trend of our survey’s findings.  Each of these systems should appeal to a certain user base.

For those who want to compare the performance of a specific system to the needs of certain applications the 157-page survey report provides a good starting point, since it breaks this information down by a number of application types and provides significantly more information than would fit into a short blog post like this one.  It can be a useful tool for both users and providers of fast storage systems, and can be purchased for immediate download from Objective Analysis at the link above.