For quite some time, embedded systems and server designers have relied on a fairly simple memory hierarchy. They’ve used volatile memory (SRAM and DRAM) for fast data and code storage close to a processor and rotating mechanical memory (hard disk drives or HDDs) for large-capacity, non-volatile storage. As processors got faster, the performance gap grew larger between processor bus cycles times and DRAM cycle times so system designers started to employ fast SRAM as a cache to alleviate some of the access-time problem with DRAM.

There’s a similar but even larger performance gap between the access time of DRAM and HDDs. Although vendors have improved HDD capacity by 60% per year—each and every year—and HDD’s price per storage bit directly tracks that trend as well, there’s been very little improvement in HDD data transfer rate and interface speed and there’s been no dramatic change in HDD access time, which is largely determined by mechanical factors. Consequently, there’s been only a relatively slow improvement in HDD IOPS (I/O operations per second), which leads to a massive five-orders-of-magnitude (10^5) performance gap between DRAM access times and HDD access times and that performance gap is growing.

At the same time, DRAM’s volatility plays a role in a system’s sensitivity to power glitches and losses. When data is critical, and most data is critical these days, non-volatile memory just isn’t sufficient. Some means of retaining data through a power loss is usually required. In the past, HDDs have sufficed for non-volatile storage but they’re simply too slow these days.

The Flash Zone

Flash memory is a good candidate for filling this memory gap because it provides nonvolatile storage and it has become the cost-per-bit leader in semiconductor memory. Consequently, Mark Gogolewski, Denali Software’s CTO, calls this performance gap in the memory hierarchy the “Flash Zone” (see figure below and the Reference), the performance zone between DRAM and HDD access times.

The Flash Zone in memory hierarchy

The reasons for Flash memory’s candidacy to fill this gap include:

• In 2004, the per-bit cost of NAND Flash dropped below the previous category leader, DRAM.
• More NAND Flash bits shipped in 2005 than bits of any other type of semiconductor memory.
• More NAND Flash bits shipped in 2007 alone than all of the DRAM bits shipped in the last 25 years of commercial DRAM production.

There’s been a huge decrease in the per-bit cost of NAND Flash and a big capacity increase on a NAND Flash die. Consequently, NAND Flash memory fits nicely in the gap between DRAM and HDD. It offers faster access speeds than HDDs by at least two orders of magnitude while replicating an HDD’s non-volatile storage abilities. In addition, NAND Flash memory can draw considerably less power than HDDs when managed correctly. The opportunity for innovation in memory hierarchy is therefore huge.

Three Ways to Use NAND Flash: SSDs, Flash Cache, DRAM Backup

There are three ways to fill the Flash zone. The first approach is to use NAND Flash memory to create an HDD emulator using the same disk interface and possibly even the same form factor. Such drives are called solid-state drives (SSDs) and they have been gaining traction in the industry. Because they do not employ rotating memory, SSDs can deliver far faster access times than HDDs. However, there are costs associated with this approach. SSDs cost substantially more per stored bit than HDDs while retaining the overhead associated with HDD interfaces and protocols. The memory bus protocols and interfaces used to connect DRAMs to processors are much, much faster.

At this time, most analysts agree that NAND Flash memory will not overtake HDDs in cost per bit. Jim Handy of Objective Design presented the chart shown below at MemCon 2008 showing that the 25x cost-per-bit advantage for HDDs relative to NAND Flash memory cost per bit would continue for the foreseeable future.

NAND Flash and HDD cost-per-bit forecasts
(Jim Handy, Objective Design)

Denali’s memory market analyst Lane Mason recently commented that the pace of cost-per-bit reductions for NAND Flash memory will actually slow compared to price drops in recent years. So it doesn’t appear that NAND Flash will supplant HDD storage in the near- or medium-term future.

The second way to use NAND Flash memory in the Flash Zone is called a Flash cache. A Flash cache speeds access to an HDD by buffering the data stream between a processor and the HDD. Data is drawn from and written to HDDs as needed and the same data is simultaneously cached in NAND Flash. The next time this data is needed, it’s drawn directly from the Flash cache instead of the slower HDD. Flash caches do not require as much NAND Flash memory as SSDs, and therefore cost less, but they can deliver performance improvements when paired with HDDs.

The third way to use NAND Flash memory is to implement a backup strategy that allows the DRAM to operate normally when system power is available and to quickly save that data in non-volatile NAND Flash when system power fails. In this approach, which is used in AgigA Tech’s AGIGARAM Non Volatile System (NVS) modules, a backup power source provides the energy needed to safely tuck data away in non-volatile storage (NAND Flash), which then retains the data for a decade or more if needed.

This third approach to filling the Flash Zone offers several benefits including:

1. Fast backup when power fails
2. No energy required to save the data during power failure
3. Automated backup and restoration of data with no host-based software assist required

Which of these three approaches to use depends on the application (as always). If you’d like help deciding, please feel free to contact AgigA Tech.

Reference

The World is Flash: A Disruption of the Memory & Storage Hierarchy, Keynote Speech, Denali Memcon 09, Mark Gogolewski, CTO, Denali Software, Inc., www.denali.com