Back in August, at the FMS show, keynoters surprised their audiences with promises of upcoming 100 million IOPS SSDs. Given that today’s highest-performing SSDs provide well under ten million IOPS, how will SSD makers reach this very challenging goal?
We can gain some insight into this from a little information in a paper published this past November for OCP 2025 by researchers at ScaleFlux, Nvidia, and Stanford University: From Minutes to Seconds: Redefining the Five-Minute Rule for AI-Era Memory Hierarchies.
The paper illustrates that NAND can come closer to being used as a memory layer than as a storage layer in the memory-storage hierarchy when the SSD’s NAND is managed differently than is done in modern SSDs. Part of this is due to the fact that certain widely-accepted beliefs might need reconsidering, and the rest comes from fundamental architectural changes to the SSDs.
Reconsidering Widely-Accepted Beliefs
The researchers challenge of one widely-accepted belief is reflected in the title of their paper. For those who haven’t run across it, Jim Gray’s 5-minute rule was first devised in the 1980s and has seemed to stand the test of time ever since. People run updated numbers through Gray’s formula, using HDDs, SSDs, and various sorts of memory, and it always seems to produce a result similar to that found in the first paper.
Gray’s formula is a very simple calculation, based on hard drive (HDD) and DRAM prices and datasheet maximum performance, and it finds that you should keep data in memory (DRAM) if it’s going to be needed within 5 minutes, but data that’s unused for 5 minutes or longer should be moved out of memory and into storage. The authors of the new paper noticed that Gray’s equation may have been just a little bit too simple, and they added in several other important factors, including wasted server resources and the typical behavior of SSDs and DRAM, and these shed a completely new light on the matter. It’s an interesting read.
Two other widely-accepted beliefs are that memory is memory and storage is storage, and that hardware and software should follow established delineations and cannot be tuned to work together as well as possible.
The new paper starts with the assumption that the researchers should consider redesigning anything that can be redesigned. This includes hardware as well as software, and the delineation between memory and storage, and it doesn’t simply assume that today’s SSD designs are the optimum for AI applications. Instead, they decided to use hardware/software co-design to optimize both sides to the benefit of the overall system, while employing a dual-resolution data structure to provide approximate results initially, on a data subset, before refining the results by evaluating the full-resolution values of only the most likely candidates. Finally, they chose to not really think of SSDs as storage, but instead consider them a slower alternative to DRAM, using flash to provide cheaper fast data access than DRAM could provide.
A New SSD Architecture
This approach made the researchers to explore ways to make SSDs with faster access, so that its performance was more closely matched to the DRAM it replaced.
They decided not to limit their exploration to today’s SSDs, but chose to look at another architecture, the Storage-Next SSD that is currently being co-developed by Nvidia, SSD suppliers, and other companies including Marvell. This SSD uses a design that is optimized for smaller transfers than the 4k-byte blocks standard to SSDs, since a new algorithm shrinks the size of the data that the GPU manipulates to reduce the computational workload.
Marvell explains that today’s SSDs don’t scale IOPS for data transfers smaller than 4KB, and this causes the PCIe bus to be underutilized, starving the GPU and wasting cycles. The current approach to this problem is to deploy a number of SSDs in parallel, which increases both the system’s cost and power, but even this approach has difficulty meeting performance targets.
Marvell explains that the limited performance of 4kB-optimized SSDs is mainly the result of their error correction algorithm. One of the Storage-Next SSD’s architectural changes is to optimize error correction for smaller data transfers.
In a recent presentation Nvidia showed a conceptual chart to illustrate how a Storage-Next SSD would perform in comparison to the industry’s current highest-IOPS SSD:
Note the caption: “Algorithm and data structure co-design for IOPS efficiency.” this co-design involves using small 512B representations of larger pieces of data, and performing initial identification of those, choosing which are closest to the target value. Once this step is complete then the processor performs a match on much wider data, a 6KB original vector, but only for those records that passed the first screening. Today’s approach compares every 6KB record using their full data width. While the new approach requires two steps, one for the 512B mini-representations of the data and one for a subset of 6KB original vectors, the net result is significantly faster processing using only the items that are the most likely fit, and ignoring all the rest.
In short: All the software must be reconfigured, and the matching process must be performed twice, once at low precision and once at full precision. By doing this, and by using an SSD optimized for 512B transfers, the cost/performance of the system can be dramatically improved.
I should note that other factors also help. Part of the Storage-Next SSD’s speed advantage comes from the fact that it will use the PCIe 7.0 interface. The rest is provided by a re-architecture of the SSD, including the error correction for smaller transfers.
There is also talk of using speed-optimized NAND flash chips. This is quite a departure from the industry’s current direction, since NAND chips are nearly always cost-optimized at the expense of speed. Some of the leading NAND flash makers have demonstrated speed-optimized flash chips for the past decade or more, particularly Samsung with its Z-NAND and SK hynix’s 4D NAND. Two tricks these chips use is that they are based on SLC flash, which is significantly faster than MLC, TLC, or QLC flash (see this post’s first chart), and these special SLC chips have a larger number of internal planes. Planes are like miniature flash chips within a single flash chip, and they improve performance, but increase the die size and therefore cost. The trick is to find the right balance of cost/performance to optimize the total system’s cost/performance. I’ll leave that to the data center designers.
One of Objective Analysis’ strengths is to understand and identify potential winning new technologies before they become big. The Storage-Next SSD is an example of a technology that has the potential to significantly change the SSD market. If you’d like to benefit from our understanding of new technologies please contact us to explore ways that we can work together to strengthen your corporate strategy.