This post is the second part of a four part series in The SSD Guy blog to help explain Intel’s two recently-announced modes of accessing its Optane DIMM, formally known as the “Intel Optane DC Persistent Memory.”
Memory Mode
The most difficult thing to understand about the Intel Optane DC Persistent Memory when used in Memory Mode is that it is not persistent. Go back and read that again, because it didn’t make any sense the first time you read it. It didn’t make any sense the second time either, did it?
Don’t worry. This is not really important. The difficulty stems from Intel’s marketing decision to call Optane DIMMs by the name “Intel Optane DC Persistent Memory.” Had they simply called them “Optane DIMMs” like everyone expected them to then there would have been far less confusion. That sentence above would have instead said that Optane DIMMs are not persistent when used in Memory Mode.
Readers would then have said: “Well, OK. But why use Optane, then, if it’s not persistent?” The answer is very simple: Optane DIMMs are enormous! Consider the fact that Samsung’s largest DRAM DIMM (a very costly one) is 128GB, and Intel’s smallest Optane DIMM is 128GB and should sell for a fraction of the price; this gives you very good reason to use Optane. Everybody wants more memory!
So why in the world is it not persistent? The answer is involved, but relatively simple to understand.
Optane cannot be used as the only memory in a system – it has to be accompanied by DRAM. This is because Optane doesn’t like to communicate with the processor the way that the processor likes to be communicated with. The module is pretty slow compared to DRAM for three reasons:
- The medium, 3D XPoint Memory, writes more slowly than it reads. Some say that a write takes three times as long as a read. If this chip were to communicate with the processor over a standard DDR4 interface then all reads would have to be slowed to the same speed as writes.
- Another difficulty is that 3D XPoint Memory wears out, so it has to use wear leveling. That means that address translation must be inserted into the critical timing path, slowing down every access, reads as well as writes.
- The third reason is one you probably never thought of: The data must be encrypted before it is stored and decrypted when it is read, further slowing the critical path. Many organizations worry that storage (HDDs, SSDs, and now NVDIMMs, including the Optane DIMM) will fall into the wrong hands making data available to evildoers. Those organizations would not use the Optane DIMM if it did not support data encryption. (Alert readers will object that this can only be an issue if the Optane DIMM is, in fact, persistent, and they’re right. I’ll explain that shortly.)
The solution is to do what all cell phones do, which is a very similar technique that has been used for decades to manage data between DRAM and HDDs or SSDs. In a cell phone the processor can’t efficiently communicate with the NAND flash, so it moves lines of code and data from the flash into a DRAM and operates on them there. In Intel’s new Memory Mode the processor moves data back and forth between the Optane DIMM and the system’s DRAM, and only executes code or operates on data in the DRAM.
The Optane DIMM is paired with a DRAM that behaves as a cache, and, like a cache, it is invisible to the user. You heard that right – if you use the Optane DIMM in Memory Mode then your DRAM becomes inaccessible. A typical system might combine a 64GB DRAM DIMM with a 512GB Optane DIMM, but the total memory size will appear to the software as only 512GB. This is the same thing that cache memory does: The size of the cache is not added to the size of the DRAM, it’s simply invisible. In either case the faster medium (the DRAM in this case) temporarily stores data that it copied from the slower medium (the Optane DIMM in this case) and the cache controller manages the data’s placement in a way that makes it appear that the Optane DIMM is as fast as the DRAM. At least, it appears that way most of the time. In those rare instances where the required data is not already in the DRAM, the data accesses slow down a lot. This is because the processor stops everything and moves data around. If necessary it copies modified DRAM data back into the Optane DIMM, and then it copies the missing data from the Optane DIMM into the DRAM. This usually occurs very rarely (maybe 1-5% of the time, depending on the software that’s being run) so the other 95-99% of the time the system will run at DRAM speeds. That’s close enough for most people.
If you want a really deep dive into this you can order The Cache Memory Book, which dissects all of the principles of caching. I happen to know, because I wrote it.
So let’s talk about persistence. Nothing has been written into persistent memory until it actually reaches the Optane DIMM. The software thinks that it’s writing to the Optane DIMM, but it’s actually writing into the DRAM cache. When there’ a surprise power outage the data in the DRAM cache vanishes. This is why Intel’s Memory Mode is not considered persistent. The data that was in DRAM waiting to be written into the Optane DIMM is lost. If some of the data is persistently stored in the Optane DIMM, but some hasn’t been updated, and if nobody knows what data has missed being written into the Optane DIMM, then all of the data is suspect. The easy answer is to say that none of the data persisted – just start over. That’s what existing DRAM-only systems assume, so it’s not an alien concept. If it’s in memory (DRAM or Optane) and the power is lost, then the data is lost as well.
So it’s not considered persistent.
I have been told that the processor memory controller could have been designed to flush all of the “Dirty” (new) data in the DRAM cache back into the Optane DIMM when power fails, thus making it fully persistent, but since the whole point of Memory Mode is to make existing software see a giant memory space without any modification, this was considered unnecessary.
Anyone who wants to take advantage of the Optane Memory’s persistence will need to use it a different way, and that’s the subject of the next post. This mode is called App Direct Mode, and it not only supports persistence, but it also allows the user to access the DRAM as DRAM (without hiding it) and the Optane Memory as Persistent Memory, so a system with 64GB of DRAM and 512GB of Optane Memory will appear to have 64+512=576GB of some kind of memory.
Just to complicate things, the memories (DRAM and Optane) don’t both have to be entirely dedicated to either Memory Mode or App Direct Mode. The software can determine just how much of either memory type will operate in Memory Mode and how much will work in App Direct Mode.
Is that confusing enough? I suspect that very few programs will manage the memory in both modes. At least, not for a very long time!
But at least you now understand that sentence at the top of this post: That Intel Optane DC Persistent Memory when used in Memory Mode is not persistent!
A I said in 2015 when I published the industry’s first 3D XPoint forecast, Memory Mode should account for the bulk of Optane’s early sales, because it can be used with existing software with no modification whatsoever. Later on, when software that uses App Direct Mode becomes generally available that should change, but this will take a number of years.
This four-part series, published in early 2019, explores each of Intel’s two modes to explain what they do and how they work in the following sections:
- Overview
- Memory Mode
- App Direct Mode
- Wrap-Up, Comparing the Modes
Your explanation of why the data is not written to persistent memory when a power outage occurs makes no sense to me.
Roger,
Thanks for the comment. I wish you had explained what part of the process is hard to follow.
In brief, some data is stored in DRAM with an expectation that it will eventually be written back into the Optane DIMM.
The processor’s Memory Controller keeps a list of the DRAM addresses which haven’t yet been written to the Optane DIMM.
During a power failure, if the data were to be immediately moved from DRAM to the Optane DIMM, then the processor would be responsible for moving all of that data, so you would need backup power for the processor, the DRAM, and the Optane DIMM for a long enough period to account for the worst-possible case – where every single DRAM line needed to be saved. That’s a lot of power!
Servers could do this today, moving all of the DRAM data into an SSD, as long as they had an even larger backup power source. They don’t because it’s uneconomical.
Does that make sense?
Jim
Maybe I’m wrong, but in my understanding in this memory mode the Optane basically will work as a hw assisted, fast “swap” drive.
Zooly, you are correct.
My cache background makes me look at everything as a cache. Your comment approaches the question the same way, but you are looking at everything as pages in a demand-paged virtual memory system.
Caches and demand paging use the same basic approach: Use something small and fast to make the big slow thing look really fast, or conversely, use something big and slow to make a small fast thing look really big.
You do that because slow things are cheap and fast things are expensive.
Compared to DRAM, an Optane DIMM looks big and slow. Compared to an SSD an Optane DIMM looks small and fast.
A swap drive makes a DRAM look really big. So does an Optane DIMM, but the Optane DIMM is faster.
In short, you are not at all wrong!
Thanks for the comment,
Jim
Your last comment is what excites me the most about all of these new technologies. What I see is an opportunity to completely rework and replace how we currently subdivide our memory and storage subsystems.
Rather than needing 32-64gb of fast ram for ram hungry applications, we instead use half and half. Half fast ram half big ram.
I guess my question is how much will this actually benefit the average consumer workload? I am mostly a gamer but also do some video/photo editing and some VMware work. The advantages for VMware are clear and obvious, a little less so for video and photo editing (but still useful in edge cases) but im not sure I see how much it might help for a PC gamer.
PC gaming seems to have pretty obvious tiered “breakpoints” where more helps and then adding more has almost no extra benefit. Currently 8gb of ram works, while 16gb helps in a fair number of situations. But moving to 32gb+ rarely does much of anything. Going from a sata3 SSD to a large NVMe SSD helped more than I expected but it was still pretty minor. The optane memory in SSD mode might give a small boost but i think the optane memory in memory mode might actually be detrimental to a gaming focused machine because it would likely reduce your ability to overclock the DRAM specific modules (something that can actually have a small effect on a gaming workload, especially on the AMD systems).
Anyway, interesting articles, thanks for posting them!
Ben,
Thanks for a very interesting comment. Apologies for not noticing it until today.
The whole idea of managing the mix of fast & slow memory is interesting. I wrote a Memory Guy post about it that explains a concept that most folks find difficult to understand: Adding a lot of slow memory/storage can improve performance more than adding a little bit of fast memory that costs the same amount.
https://TheMemoryGuy.com/why-dram-is-threatened-by-ssds/
I wish there were software tools that would help people choose the best mix of DRAM & SSD for their particular application and hardware, but there doesn’t seem to be anything like this.
If a tool like this were to provide you with the best mix of DRAM, Optane, SSD, and HDD everyone would have a really easy time making the kinds of decisions that you describe. Of course, you would have to tell it what software you use, your CPU type, and a lot of other things (like how much money you want to spend).
Maybe someone will make a tool like this someday.
Thanks again!
Jim
Is it possible to use only optane dc dimms in all dim’s channels instead of native dimms (i know the performance will be degraded)?
Yan, That’s a good question.
The answer is no. You MUST have at least one DRAM DIMM per channel.
In Memory Mode this DIMM caches the Optane DIMM’s contents. Optane’s contents are managed into and out of the DRAM by the MMU (Memory Management Unit) and aren’t directly accessible by the CPU.
In App Direct Mode Data can be managed into and out of the Optane DIMM, but code cannot be executed from it.
Even with this restriction, you can still get a whole lot more memory into a couple of DIMM slots with DRAM-plus-Optane than you could with only DRAM, though, and it’s cheaper.
Hope that helps,
Jim
The memory mode of DCPMM uses DRAM to cache data from NVM. In this mode, can operating system see DRAM and how data is evicted?
The next question is when we use the MIXED MODE,the DRAM also invisible to user.Will it still work on even if the portion of MEMORY MODE is 1% and the least is 99% APP DIRECT MODE?
Finally,I conceive a strategy that use APP DIRECT MODE but act like MEMORY MODE.If I configuration the DCPMM as APP DIRECT MODE,then we have two area,DRAM area and APP DIRECT MODE(ADM for short) area.Now I try to use the entirely AMD area as system memory and the DRAM area as operational part,then I can manual achieve LL4 system memory.How much performance different between the MEMORY MODE and the conceive strategy?
If possible,Is there more specific document?thanks!
Daynes, Sorry I didn’t notice your comment until now. Here’s a very late reply.
Your question is long and complicated. I will do my best to answer.
Let me answer your last question first. I am sure that there is significantly more information on Intel’s website, but it’s not all in one place. I suggest you spend some time to have a look.
For your first question, Memory Mode is made to be invisible to the operating system, so the O/S doesn’t really know which data is copied into DRAM and which is only in the 3D XPoint DIMM. The O/S won’t know how the data is evicted.
In mixed usage you can think of the system as if it were two completely different parts. The areas that are mapped for App Direct Mode will work independently from the areas are mapped for Memory Mode. I guess you could even have an application program that was managing its persistent data in App Direct Mode and map certain memory areas into Memory Mode (for example the code space), but that would be more complicated than anything that I have heard of so far. So, YES, a 1% Memory Mode and 99% App Direct Mode system should work.
I am not sure that I understand your third paragraph (the longest one). First of all, is LL4 a combination of L4 and LLC (4th-level cache and last-level cache)? Second, when you say that you would use the App Direct area for system memory and the DRAM area as operational, I believe that you mean that the App Direct portion would not be available to the application. That is the opposite of the intention of App Direct Mode. The purpose of App Direct Mode is to give applications access to persistent memory.
Finally, you ask what the performance difference would be between your strategy and Memory Mode. This will depend entirely on your application. Certain applications desperately need persistence and perform a lot of SSD writes to guarantee that data has been put somewhere safe. These applications will be accelerated considerably once they start to use App Direct Mode. I would not be surprised to hear of an application that was 100X faster with Optane DIMMs in App Direct Mode than in an SSD environment. But since your concept does not appear to be aimed at such applications then I would not expect to see any improvement over standard Memory Mode.
It appears that you think that the Optane DIMM understands whether it is being used in Memory Mode or App Direct Mode – it does not. An Optane DIMM knows nothing about how it is being used. That understanding is part of the memory management software in the operating system.
I hope that helps.
Jim
“Consider the fact that Samsung’s largest DRAM DIMM (a very costly one) is 128GB, and Intel’s smallest Optane DIMM is 128GB and should sell for a fraction of the price”
This statement is very misleading. Optane DIMMs can only occupy one of the two DIMM slots of a memory channel, while normal DIMMs can occupy both channels. This means that a server can have twice as many DIMMS than Optane DIMMs. Everyone who writes about Optane DIMMs seems to make this mistake.
Also, comparing Intel’s listed Optane DIMM prices to current DIMM memory prices, they are in fact *more* expensive.
Edu, Interesting points.
Intel’s plan is for everyone to use Optane in only one of the two DIMM slots and to use DRAM in the other. The DRAM either caches the Optane data or it can serve as an independent memory that is faster than the Optane. Intel says that you shouldn’t try to use only Optane in any DRAM channel, but Baidu has done that successfully. I suspect that they had to use custom software. This means that Baidu was able to put Optane onto BOTH of the channel’s slots.
Regarding pricing: You compare Intel’s list price against DRAM module prices on the Internet. It would be more fair to compare Intel’s list price against the list price for DRAM modules. List prices are usually much higher than those on the Internet. The prices I have used on The Memory Guy blog are simply the ones that Intel used for their price/performance comparisons.
Thanks for the comments,
Jim