An NVDIMM Primer (Part 2 of 2)

This post is the second of a two-part SSD Guy series outlining the nonvolatile DIMM or NVDIMM.  The first part explained what an NVDIMM is and how they are named.  This second part describes the software used to support NVDIMMs (BIOS, operating system, and processor instructions) and discusses issues of security.

Software Changes

Today’s standard software boots a computer under the assumption that the memory at boot-up contains random bits — this needed to be changed to support NVDIMMs.  The most fundamental of these changes was to the BIOS (Basic I/O Subsystem), the code that “wakes up” the computer.

The BIOS is responsible for detecting all of the computer’s hardware and installing the appropriate drivers, after which it loads the bootstrap program from the mass storage device into the DRAM main memory.  When an NVDIMM is used the BIOS must understand that the data within the NVDIMM may already be valid and should not be over-written, and that the NVDIMM may need some time to move data from the on-DIMM flash to the DIMM’s DRAM.  In the event of a power failure the BIOS also assumes the responsibility of storing the state of all of the processor’s registers, along with the “dirty” lines of the processor’s cache, into the NVDIMM.  The register status must then be restored at power-on.

But this is all housekeeping, and doesn’t take full advantage of the NVDIMM’s speed advantage over an SSD or HDD.  More software needed to be developed to prevent NVDIMM accesses from being bogged down by slow I/O routines that were developed for HDD and SSD.  At the time that they were written these routines were significantly faster than the drives themselves.  With NVDIMMs the opposite is true: The storage hardware (the NVDIMM) is significantly faster than the I/O routines accessing it.

The Storage Networking Industry Association (SNIA) has taken a leadership role in defining a standard software structure for communicating with a broad set of SCM types, which SNIA calls “Persistent Memory”.  This allows operating system developers to create standard calls to access the persistent memory of an NVDIMM.  These standard calls allow applications programmers to write portable applications software that can take advantage of persistent memory across a number of platforms without any custom redesign.

Last June SNIA released a revised version of its NVM Programming Model (the above diagram) which adds operating system calls that applications can use to access local (left) or remote (right) persistent memory either through the I/O block protocol or as memory.  Most importantly, the application program knows which memory accesses are persistent and which ones are not.

Processor Architecture Upgrades

The use of NVDIMMs not only impacts software, but it also has repercussions on processor architecture.  In today’s systems a power failure is treated as a total loss.  The contents of DRAM are lost along with any dirty cache lines and the contents of all of the processor’s registers and write buffers.  With NVDIMMs this doesn’t have to be the case.

Intel added new instructions to its standard instruction set to help move the processor’s entire state into an NVDIMM in the event of such an emergency.  This required the addition of only three new instruction classes which are documented in Intel’s August 2015 update of the Intel Architecture Instruction Set Extensions Programming Reference.

All of Chapter 10 of this reference is devoted to these “Memory Instructions” even though only three new instructions were added:

• CLFLUSHOPT: An “optimized” flush of a dirty cache line that invalidates that cache line
• CLWB: The same thing but without invalidation
• PCOMMIT: “Commits” an NVDIMM’s DRAM (or iMC write buffers) to its back-up NAND flash

The first two new instructions take care of concerns that cached data may be more current than its main memory equivalent, by assuring that the data stored in the NVDIMM after a power failure should be the most current rendition.

The third instruction allows the processor to initiate the process of copying an NVDIMM-N’s DRAM contents to its NAND, rather than relying solely on hardware control.  This instruction would not be used with those NVDIMMs that are built exclusively of nonvolatile memory and do not use a DRAM.

Security Issues

Since NVDIMMs are persistent they are subject to the same security risks as HDDs and SSDs: Someone might steal your HDD, SSD, or NVDIMM and all the data that’s on it.

Over the past few years Self-Encrypted Drives (SEDs) have become widely available, providing SSD and HDD users a way to ensure that their data will not fall into the wrong hands even if the SSD or HDD is stolen.  So far NVDIMMs don’t offer such security.

SNIA is, once again, deeply involved in this, and is currently working on a white paper to discuss this issue and potential solutions.  As this post was being written the white paper consisted of little more than an outline and a table that outlines the “Threat Model” as SNIA seeks input from its members as well as from the general public.

Unlike SSDs and HDDs, the very low latency of an NVDIMM makes standard encryption engines difficult to use, since they would add significant latency to the device.  Over time alternate approaches to support NVDIMM encryption are certain to be developed.

Another potential problem arises out of the fact that an NVDIMM resides within fixed address range while the system’s standard DRAM resides in another fixed address range.  This works against a hacking countermeasures like “address space layout randomization”, which is used in operating systems’ kernels to randomize where code is stored in memory.  If the code moves around in the memory then it is very difficult for a hacker to insert malicious changes.  Since NVDIMM resides at a fixed address range the addresses will be less random than they would be in a system without NVDIMMs.

For More Information

Objective Analysis is deeply involved in the NVDIMM market as well as with SSDs and memory chips.  We offer our clients considerable insight into these products from the perspectives of both marketing and technology.  We specialize in helping our clients plan for the future to achieve success.

If your company would like to take advantage of our insight and deep understanding of this  market please visit the Objective Analysis website to learn the business models we use to support our clients.