The question’s not as simple as it seems. There’s a temporal quality to the question. What’s right ten years ago isn’t right today and probably won’t be right ten years from now. Semiconductor technology is both fluid and extremely dynamic. One thing’s certain. You need to deal with today’s problems today. If you can address the same problem in the same way two or three years from now, that’s great! But you still need to address today’s problem today. You need to use components you can get today, not some time in the future. The future may include some surprises that change today’s answer, but today’s answer must be based on what you can do today.

Why the emphasis on today? Well, any RAID server memory used today must be based on some sort of memory technology (or technologies) that’s commercially viable now. Researchers are working on more than a dozen new memory technologies that may someday produce a more ideal memory than the semiconductor memories we have at our fingertips today. It’s not clear when that might happen. Tantalizing technology announcements are made almost weekly. But technology announcements are generally light years away from being commercially competitive products and that’s never truer than when you’re talking about digital memory.

Bulletproof RAID server memory must have some mechanism to ride through power outages without data loss.  The previous two entries in this series (Part 1 and Part 2) discussed various approaches to creating bulletproof memory using battery-backed RAM. Seems like a great idea, but batteries aren’t particularly reliable in data-center environments where they live inside of heat-generating boxes squeezed into rack upon rack upon rack where they get no light and precious little maintenance. High-maintenance components like batteries just seem like a poor choice for creating memory that’s supposed to be bulletproof. Wouldn’t you agree?

So what’s that leave?

Well, you could use NAND Flash for memory rather than DRAM. NAND Flash devices have many excellent attributes. They do not require power to provide nonvolatile storage. They are currently the semiconductor industry’s cost-per-bit leader. NAND Flash chips available in higher capacities than DRAMs, which translates into more bits per same-size board, fewer devices per board for same-size capacity, or smaller boards depending on application needs. These are all great attributes.

Unfortunately, NAND Flash devices have some unhappy qualities as well. You can only write to them relatively slowly—much more slowly than DRAM. They also exhibit wearout failure, which is getting to be a bigger and bigger problem as lithographies shrink. NAND Flash devices are block oriented so you can’t write just one word. These three failings are major and make NAND Flash memories unsuitable for RAID server memories.

Unsuitable, that is, when used alone.

However, volatile DRAM paired with non-volatile NAND Flash make a pretty good team when it comes to building bulletproof RAID server memory. When the power’s good, use the DRAM like…well…DRAM. When there’s an indication that power’s about to fail, save the contents of the DRAM in NAND Flash devices.

Note that you can’t let the host CPU save the data when power’s already on the slippery downhill slope. You really don’t know how much time there is before the host CPU loses its mind. You need something more—bulletproof. You need a backup power supply that will sustain the memory subsystem during the data-backup operation and you need a local processor to oversee the transfer.

Batteries are still bad

The previous two installments of this series have already dealt with the many reasons that batteries are not suitable as the backup power supply. Barring the sudden invention of the Mr. Fusion portable reactor last seen attached to the back of Doc Brown’s DeLorean time machine in the Back to the Future movies, there’s really only one good alternative for emergency backup power for RAID server memories: ultra-capacitors.

Ultra-capacitors are capacitors that have electrodes with greatly expanded area, which result in greatly expanded capacitance. The electrode area expansion originates in porous carbon electrodes. Ultra-capacitors have capacities measured in Farads, much greater then conventional electrolytic capacitors. Although they require the proper care when designed into a backup power supply, ultra-capacitors can provide enough backup energy to support the emergency transfer of data from DRAM to NAND Flash memory in a bulletproof RAID server memory subsystem.

How practical is all this? Very practical. Take a look at the following graph, which plots projected memory costs in dollars per megabyte over the next few years. (This graph is based on iSuppli projections.)

As you can see, DRAM and NAND Flash are the least expensive semiconductor memories, per megabyte, and a megabyte of NAND Flash costs about one tenth of what a megabyte of DRAM costs. All of the leading “future” memories, which may someday replace DRAM, cost more. Some cost much more and they will continue to cost more into the foreseeable future. These “future” memory technologies are not about to replace DRAM today or tomorrow. They cost too much.

Finally note the dashed blue line. This line represents the per-bit cost of AGIGARAM, which fuses DRAM, NAND Flash, and ultra-capacitors to create the closest thing to a bulletproof RAID server memory that you can get today. Over time, the cost of a megabyte of AGIGARAM approaches the cost of the equivalent amounts of DRAM and NAND Flash added together. The cost of the memories will essentially dominate the other costs (controller, ultra-capacitor backup power source). Consequently, AGIGARAM, which is AgigA Tech’s bulletproof memory for RAID servers that’s available today, is not only the best technical approach to creating bulletproof memory, it’s the most cost-effective approach available today…and tomorrow.

In fact, one of those batteries has failed. The RAM cache it protects is at risk when the next power outage occurs. When that happens, one or more of the data center’s customers will lose data. Critical data. After all, what data isn’t critical?

Worse, the failed battery is leaking. Acid is oozing out of the battery. It’s quite possible that the acid is leaking onto critical circuitry inside of the RAID enclosure. Drip. Drip. Drip. The acid starts to etch into the circuitry. The disaster is perhaps moments away…

Customers buy one thing from RAID vendors: a safe haven for their bits. The bulletproof aspect of a RAID array’s disk storage resides in the redundancy of the disk drives themselves. A RAID 5 array protects against data loss should one disk drive fail and a RAID 6 array protects against faults should two drives fail. Both types employ disk striping with parity (double parity for RAID 6). Because data has value—and some data has tremendous value—the use of RAID systems based on hardware RAID controllers is skyrocketing. However, power loss can negate the efficacy of a RAID system and puts the data at risk.

One critical point of failure in RAID systems with respect to power outages is the write cache. RAID systems employ write caches to speed disk transactions—to boost the IOPS (I/O operations per second) rating. Once a computer system squirts a chunk of data into a RAM-based cache, the RAID system can immediately acknowledge the transaction before actually writing the data to disk. So there’s a critical period of time when the data is at risk from a power failure, after the acknowledgement but before the data is on the disk. If power is lost while the data is in RAM cache, then it’s lost forever.

One way to avoid this problem entirely is to disable the RAID system’s RAM cache. This approach preserves the data but with a huge performance hit. No RAM cache, no performance.

Another way to avoid the problem is to protect the data in a write cache from power failures using a battery-backup unit (BBU). That way, the RAID controller can recognize an impending power failure, can halt transactions, and the BBU will maintain any data yet to be written to disk and thus ride through the power failure.

Sounds great in theory, but in practice there are many problems with BBUs:

• Batteries have short, finite lifetimes compared to other electronic components and heat further shortens their electrochemical lives. There’s heat aplenty inside most server enclosures. Consequently, battery health should be closely monitored but it’s often not monitored at all. In fact, some data-center operations teams are surprised to discover that there’s a high-maintenance battery inside of many RAID systems. Of course, by the time they realize that there’s a battery to be maintained, it’s often too late because the event that brought this fact to light was a data failure induced by power loss.

• Batteries need to be replaced every one to two years. First, that’s not going to happen if no one knows there’s a battery to be replaced. Second, battery maintenance often falls pretty low on the priority list of tasks to be performed and the replacement may be dangerously deferred when it’s done at all. Third, there’s no standardization in BBUs so the correct battery pack may not be on hand. Worse, the required BBU may be discontinued, no longer be available. If you can’t order a new one, then what? Fourth, battery packs cost money and so does the time it takes to install new ones.

• When replacing the BBU, the RAID server must be taken offline, or at least the RAM cache needs to be taken off line and it must stay off line until the BBU charges up. RAID performance suffers during the downtime. Consumer-level products such as PCs and PVRs (personal video recorders) may not benefit much from faster disk drives. Enterprise systems do. Enterprise computing clients know precisely what a second’s worth of delay costs in their business. Sometimes a microsecond’s delay costs big money. For example, Google and Amazon know to the penny what each additional second of response delay costs them in terms of lost customer purchases. High-frequency securities traders and arbitrage houses employ trading strategies that are highly dependent on ultra-low latency networks. In fact, they co-locate their trading servers with the trading floor to minimize communications latency with the computers at the market exchange. These traders profit only by feeding information on competing bids and offers to their trading algorithms microseconds faster than their competitors. Loss of write-cache performance in a RAID system could literally cost such traders millions of dollars per microsecond of delay.

• Batteries are not environmentally friendly so it’s a bad idea to just toss them in the trash. Batteries should be properly recycled and proper recycling is expensive, beyond the cost of the replacement BBU. Even when recycled properly, batteries just aren’t that great for the environment.

So what’s the right answer to the need for bulletproof RAID write cache? AgigA Tech believes that the answer can be found in a fusion of NAND Flash and ultra-capacitor technologies. Ultra-capacitors are essentially made of benign carbon and have many superior qualities compared to batteries. In particular, they charge faster (less downtime) and they have longer lives (when properly applied). NAND Flash can save a RAM cache’s contents indefinitely and without power. So AgigA Tech’s AGIGARAM modules can be used as RAID RAM-cache modules, providing all of the benefits of battery-backed write caches but without the many liabilities batteries incur.

What about the cost of such an approach? Stay tuned. We’ll address that in the next blog entry.

Of course, the culprit is you. You didn’t properly maintain the device by instituting a program of regular maintenance and periodic battery checkups. How could you? The device was tucked away on a shelf, perhaps years ago, and you forgot all about it. After all, batteries aren’t the most important thing in your life. You’re a busy person.

This same scenario applies to batteries in all sorts of embedded equipment and servers. As little electrochemical electricity factories, batteries need to be maintained or they will eventually give you a nasty surprise. It’s that simple.

Over the last 10 years or so, ultra-capacitors have started to replace batteries in a variety of electronic applications where large amounts of energy storage are needed. One of the primary uses for ultra-capacitors is memory backup. A more conventional approach to memory backup employs ultra-capacitors to provide backup power to SRAM or DRAM subsystems in the event of a power outage. However, even large banks of ultra-capacitors cannot back up memory for years. A different sort of approach to preserving data in the event of a power mains failure is to draw energy from ultra-capacitors just long enough to move critical data from volatile SRAM and DRAM into non-volatile Flash memory. Then the Flash memory can retain the data for ten years or more with no power at all.

Ultra-capacitors get their high capacities from porous carbon electrodes that provide massive amounts of surface area in tiny spaces. As nanotech research delves into the mysteries of carbon-based nanostructures such as nanotubes, ultra-capacitor storage capacities improve. Coincidentally, the number of ultra-capacitor vendors has recently been increasing and therefore the effort required to evaluate the various offerings has also been increasing.

Characterizing an ultra-capacitor isn’t simple. For long-term use in critical embedded and server systems, you need to know how an ultra-capacitor’s electrical storage abilities change over time, temperature, and voltage. It turns out that the long-term characteristics of these low-voltage devices are extremely sensitive to temperature and to operating voltage. They’re also sensitive to the way they’re charged and discharged, so the design of charging and discharging circuitry is critical to the safe, long-term use of ultra-capacitors in multiple-device banks.

If you’re designing memory subsystems and wish to use ultra-capacitors for power backup, you have two choices. On the one hand, you can mount your own ultra-capacitor characterization program and develop your own charging and discharging circuitry. Alternatively, you might choose to use a pre-designed, pre-characterized power module based on ultra-capacitors that’s specifically designed for memory-backup applications such as the PowerGEM offered by AgigA Tech. Either way, ultra-capacitors offer a good alternative to backup batteries, one well worth investigating.

References:

Energy Storage Industry Needs Novel Circuits And Semiconductors, Bobby Maher, http://electronicdesign.com/Articles/Index.cfm?AD=1&ArticleID=21973&bypass=1

Ultracapacitors Challenge the Battery, John M Miller, http://www.kilofarad.org/files/Ultracap-%20World%20&%20I%20-%20June%202004.pdf