The question’s not as simple as it seems. There’s a temporal quality to the question. What’s right ten years ago isn’t right today and probably won’t be right ten years from now. Semiconductor technology is both fluid and extremely dynamic. One thing’s certain. You need to deal with today’s problems today. If you can address the same problem in the same way two or three years from now, that’s great! But you still need to address today’s problem today. You need to use components you can get today, not some time in the future. The future may include some surprises that change today’s answer, but today’s answer must be based on what you can do today.

Why the emphasis on today? Well, any RAID server memory used today must be based on some sort of memory technology (or technologies) that’s commercially viable now. Researchers are working on more than a dozen new memory technologies that may someday produce a more ideal memory than the semiconductor memories we have at our fingertips today. It’s not clear when that might happen. Tantalizing technology announcements are made almost weekly. But technology announcements are generally light years away from being commercially competitive products and that’s never truer than when you’re talking about digital memory.

Bulletproof RAID server memory must have some mechanism to ride through power outages without data loss.  The previous two entries in this series (Part 1 and Part 2) discussed various approaches to creating bulletproof memory using battery-backed RAM. Seems like a great idea, but batteries aren’t particularly reliable in data-center environments where they live inside of heat-generating boxes squeezed into rack upon rack upon rack where they get no light and precious little maintenance. High-maintenance components like batteries just seem like a poor choice for creating memory that’s supposed to be bulletproof. Wouldn’t you agree?

So what’s that leave?

Well, you could use NAND Flash for memory rather than DRAM. NAND Flash devices have many excellent attributes. They do not require power to provide nonvolatile storage. They are currently the semiconductor industry’s cost-per-bit leader. NAND Flash chips available in higher capacities than DRAMs, which translates into more bits per same-size board, fewer devices per board for same-size capacity, or smaller boards depending on application needs. These are all great attributes.

Unfortunately, NAND Flash devices have some unhappy qualities as well. You can only write to them relatively slowly—much more slowly than DRAM. They also exhibit wearout failure, which is getting to be a bigger and bigger problem as lithographies shrink. NAND Flash devices are block oriented so you can’t write just one word. These three failings are major and make NAND Flash memories unsuitable for RAID server memories.

Unsuitable, that is, when used alone.

However, volatile DRAM paired with non-volatile NAND Flash make a pretty good team when it comes to building bulletproof RAID server memory. When the power’s good, use the DRAM like…well…DRAM. When there’s an indication that power’s about to fail, save the contents of the DRAM in NAND Flash devices.

Note that you can’t let the host CPU save the data when power’s already on the slippery downhill slope. You really don’t know how much time there is before the host CPU loses its mind. You need something more—bulletproof. You need a backup power supply that will sustain the memory subsystem during the data-backup operation and you need a local processor to oversee the transfer.

Batteries are still bad

The previous two installments of this series have already dealt with the many reasons that batteries are not suitable as the backup power supply. Barring the sudden invention of the Mr. Fusion portable reactor last seen attached to the back of Doc Brown’s DeLorean time machine in the Back to the Future movies, there’s really only one good alternative for emergency backup power for RAID server memories: ultra-capacitors.

Ultra-capacitors are capacitors that have electrodes with greatly expanded area, which result in greatly expanded capacitance. The electrode area expansion originates in porous carbon electrodes. Ultra-capacitors have capacities measured in Farads, much greater then conventional electrolytic capacitors. Although they require the proper care when designed into a backup power supply, ultra-capacitors can provide enough backup energy to support the emergency transfer of data from DRAM to NAND Flash memory in a bulletproof RAID server memory subsystem.

How practical is all this? Very practical. Take a look at the following graph, which plots projected memory costs in dollars per megabyte over the next few years. (This graph is based on iSuppli projections.)

As you can see, DRAM and NAND Flash are the least expensive semiconductor memories, per megabyte, and a megabyte of NAND Flash costs about one tenth of what a megabyte of DRAM costs. All of the leading “future” memories, which may someday replace DRAM, cost more. Some cost much more and they will continue to cost more into the foreseeable future. These “future” memory technologies are not about to replace DRAM today or tomorrow. They cost too much.

Finally note the dashed blue line. This line represents the per-bit cost of AGIGARAM, which fuses DRAM, NAND Flash, and ultra-capacitors to create the closest thing to a bulletproof RAID server memory that you can get today. Over time, the cost of a megabyte of AGIGARAM approaches the cost of the equivalent amounts of DRAM and NAND Flash added together. The cost of the memories will essentially dominate the other costs (controller, ultra-capacitor backup power source). Consequently, AGIGARAM, which is AgigA Tech’s bulletproof memory for RAID servers that’s available today, is not only the best technical approach to creating bulletproof memory, it’s the most cost-effective approach available today…and tomorrow.

There’s a similar but even larger performance gap between the access time of DRAM and HDDs. Although vendors have improved HDD capacity by 60% per year—each and every year—and HDD’s price per storage bit directly tracks that trend as well, there’s been very little improvement in HDD data transfer rate and interface speed and there’s been no dramatic change in HDD access time, which is largely determined by mechanical factors. Consequently, there’s been only a relatively slow improvement in HDD IOPS (I/O operations per second), which leads to a massive five-orders-of-magnitude (10^5) performance gap between DRAM access times and HDD access times and that performance gap is growing.

At the same time, DRAM’s volatility plays a role in a system’s sensitivity to power glitches and losses. When data is critical, and most data is critical these days, non-volatile memory just isn’t sufficient. Some means of retaining data through a power loss is usually required. In the past, HDDs have sufficed for non-volatile storage but they’re simply too slow these days.

The Flash Zone

Flash memory is a good candidate for filling this memory gap because it provides nonvolatile storage and it has become the cost-per-bit leader in semiconductor memory. Consequently, Mark Gogolewski, Denali Software’s CTO, calls this performance gap in the memory hierarchy the “Flash Zone” (see figure below and the Reference), the performance zone between DRAM and HDD access times.

The Flash Zone in memory hierarchy

The reasons for Flash memory’s candidacy to fill this gap include:

• In 2004, the per-bit cost of NAND Flash dropped below the previous category leader, DRAM.
• More NAND Flash bits shipped in 2005 than bits of any other type of semiconductor memory.
• More NAND Flash bits shipped in 2007 alone than all of the DRAM bits shipped in the last 25 years of commercial DRAM production.

There’s been a huge decrease in the per-bit cost of NAND Flash and a big capacity increase on a NAND Flash die. Consequently, NAND Flash memory fits nicely in the gap between DRAM and HDD. It offers faster access speeds than HDDs by at least two orders of magnitude while replicating an HDD’s non-volatile storage abilities. In addition, NAND Flash memory can draw considerably less power than HDDs when managed correctly. The opportunity for innovation in memory hierarchy is therefore huge.

Three Ways to Use NAND Flash: SSDs, Flash Cache, DRAM Backup

There are three ways to fill the Flash zone. The first approach is to use NAND Flash memory to create an HDD emulator using the same disk interface and possibly even the same form factor. Such drives are called solid-state drives (SSDs) and they have been gaining traction in the industry. Because they do not employ rotating memory, SSDs can deliver far faster access times than HDDs. However, there are costs associated with this approach. SSDs cost substantially more per stored bit than HDDs while retaining the overhead associated with HDD interfaces and protocols. The memory bus protocols and interfaces used to connect DRAMs to processors are much, much faster.

At this time, most analysts agree that NAND Flash memory will not overtake HDDs in cost per bit. Jim Handy of Objective Design presented the chart shown below at MemCon 2008 showing that the 25x cost-per-bit advantage for HDDs relative to NAND Flash memory cost per bit would continue for the foreseeable future.

NAND Flash and HDD cost-per-bit forecasts
(Jim Handy, Objective Design)

Denali’s memory market analyst Lane Mason recently commented that the pace of cost-per-bit reductions for NAND Flash memory will actually slow compared to price drops in recent years. So it doesn’t appear that NAND Flash will supplant HDD storage in the near- or medium-term future.

The second way to use NAND Flash memory in the Flash Zone is called a Flash cache. A Flash cache speeds access to an HDD by buffering the data stream between a processor and the HDD. Data is drawn from and written to HDDs as needed and the same data is simultaneously cached in NAND Flash. The next time this data is needed, it’s drawn directly from the Flash cache instead of the slower HDD. Flash caches do not require as much NAND Flash memory as SSDs, and therefore cost less, but they can deliver performance improvements when paired with HDDs.

The third way to use NAND Flash memory is to implement a backup strategy that allows the DRAM to operate normally when system power is available and to quickly save that data in non-volatile NAND Flash when system power fails. In this approach, which is used in AgigA Tech’s AGIGARAM Non Volatile System (NVS) modules, a backup power source provides the energy needed to safely tuck data away in non-volatile storage (NAND Flash), which then retains the data for a decade or more if needed.

This third approach to filling the Flash Zone offers several benefits including:

1. Fast backup when power fails
2. No energy required to save the data during power failure
3. Automated backup and restoration of data with no host-based software assist required

Which of these three approaches to use depends on the application (as always). If you’d like help deciding, please feel free to contact AgigA Tech.

Reference

The World is Flash: A Disruption of the Memory & Storage Hierarchy, Keynote Speech, Denali Memcon 09, Mark Gogolewski, CTO, Denali Software, Inc., www.denali.com

1. October 10, 2009. T-Mobile Sidekick owners found that they’d lost their contacts, calendar entries, to-do lists and photos when Microsoft subsidiary Danger suffered a technical glitch. At first, the news was very bad. The lost data looked unrecoverable. Then, it looked like some of the data might be recovered. Then most. If you are or were a T-Mobile Sidekick user, what would you be thinking about the service right now?

2. September 24, 2009. Google’s Gmail blows up, again. Only a “few” users are affected, but it’s the fourth time in two years that Gmail has made the news because of service loss.

3. September 6, 2009. Twitter fails for hours. Sure the Twitter Fail Whale shows up regularly, but Twitter is a high flyer with huge visibility.

4. August 3, 2009. eBay’s PayPal crashes for five hours. PayPal loses millions of dollars in transactions that don’t happen. PayPal’s merchant customers lose more.

5. June 29, 2009. Rackspace loses power in its Dallas data center and ends up rebating customers millions of dollars in usage credits for lost service.

6. January 6, 2009. Salesforce.com’s servers crash for about half an hour. One blogger notes: “Salesforce demonstrates how (un)reliable SaaS really is.”

This sort of press is a server provider’s worst nightmare. One of the missions of this blog will be to propose approaches to improving server reliability. Please feel free to contribute your ideas.