At last month’s SNIA Persistent Memory Summit Oracle presenter Jia Shi, Sr. Director of Exadata Development, shared some statistics on the Exadata system’s history over the past ten years. (Click on the graphic to the left to see the timeline.) The speaker highlighted the fact that the system’s I/O performance has grown from 0.05 million IOPS ten years ago to 16 million IOPS today, a 320X improvement! Shi said that Exadata was designed to be “the ideal database hardware with smart system software and automated management.” There’s every reason for her to be proud of her own work with this product!
The most recent iteration of the system, X8M, released last September, takes advantage of Persistent Memory (PM) in the form of Intel’s new Optane DIMMs (formally called “The Intel Optane DC Persistent Memory module”). The presenter said she was diligently working on this new approach at this time last year – so diligently, in fact, that she was unable to attend the 2019 Persistent Memory Summit even though she was working on a pioneering implementation of PM technology!
While the timeline in this post’s graphic doesn’t underscore the fact, the IOPS delivered by the latest iteration of the Exadata system were no less than stunning. I decided that this was worth charting, which I will share below:
Shi’s slide calls out the points at which the Exadata system converted from HDDs to SCSI SSDs (2009) and then to NVMe SSDs (2014). Solid improvements resulted from these changes, but the use of persistent memory last September drove a 2.5X IOPS improvement all at once. Shi observed that both the SCSI SSD and the NVMe SSD initially boosted the system’s speed, but that performance for each eventually hit a plateau. Persistent Memory has allowed the system to break through the NVMe plateau to such an extent that the bottleneck has now shifted to the interconnect, and that’s now a focus area for future upgrades.
To get the most out of the system’s PM Shi used something she calls “The Dynamic Duo”: Persistent Memory and an RDMA NIC. By using the RDMA approach to share PM the design avoids interrupting the processor for data transfers. This saves about 50 microseconds at each end of the network, shortening remote PM accesses from roughly 120μs down to only 19μs. The PM itself contributes only about 1μs to either number, so it’s easy to understand why the interconnect is such an important focus area for future Exadata improvements.
Stories like this are just starting to be told, and they help to explain why Objective Analysis predicts for the Optane DIMM market to grow to $3.59 billion by 2023 in our latest 3D XPoint Memory report. (This report can be purchased for immediate download on the Objective Analysis website.)
Jia Shi and her team have produced incredible results, which should prove invaluable to Oracle. The SSD Guy gives his congratulations to them and to all of Oracle for providing a compelling early example of the profound impact persistent memory will have on tomorrow’s computing architectures.
