Why are the solid state disk drives still so expensive?

They are on the market for years and still so expensive. SSD of a reasonable capacity (256GB) costs as much as $800 or more. Aren’t they going to drop the prices?

Although the question appears to have been posed by someone not closely familiar with the ins and outs of hard-disk drive (HDD) and solid-state disk (SSD) technologies, markets, and pricing, it’s a frequent question posed by many in the industry. We’ve become so accustomed to large, regular drops in price/capacity for both mechanical storage (“rotating rust”) and semiconductor memory that we’ve collectively developed a sense of entitlement. If we can’t buy it today, we think, surely the price will drop and we’ll be able to afford it soon.

However, when we compare the price/capacity of SSDs against HDDs, we’re comparing one moving target against another. Moore’s Law governs the price of SSDs because the largest cost component in an SSD is NAND Flash memory (see below). Moore’s Law has been a monster force in the semiconductor industry, pushing prices ever lower for more than four decades. However, the HDD vendors are constantly working with their own price-reduction curve, which has proven to be just as robust as Moore’s Law. By pulling a veritable menagerie of rabbits out of various technological hats, HDD vendors have dropped per-bit pricing for HDDs about as fast as semiconductor vendors have cut the price/bit of NAND Flash memory.

Take a look at this graph from Handy’s keynote:

From the gross slopes of the two curves, you can see that HDD cost/capacity has remained about 20x lower than NAND Flash memory cost/capacity throughout this decade. Note that in 2006, there was a serious downturn in the slope of the curve for NAND Flash. Extrapolating that new slope led some to predict that NAND Flash cost/Gbyte would cross over that of HDDs by 2008 or 2009. That just didn’t happen. The increased rate of price decline was economically unsupportable and caused huge turmoil among NAND Flash vendors. (For extensive analysis of this situation, see this blog entry on Denali Software’s Web site.)

Now please understand, the expectation that NAND Flash cost/Gbyte would zoom past the HDD cost/Gbyte curve wasn’t just wishful thinking. NAND Flash per-bit costs did overtake and then zoom past that of DRAM, which was once the semiconductor industry’s king of cost/bit. That event happened in 2004 as shown in this slide from Handy’s keynote.

So the expectation that NAND Flash cost/bit would zoom past HDD cost/bit wasn’t at all far-fetched. It just didn’t happen. HDD vendors happily continued to cut the cost/bit of rotating storage, to the very great benefit of consumers and enterprise users everywhere.

Handy’s simple silicon anatomy of an SSD shows why the SSD’s cost/bit is closely tied to the cost of NAND Flash.

From a silicon perspective, Handy’s illustration shows 34 key semiconductor devices in his example 64-Gbyte SSD. Two of the devices are a controller chip and a DRAM buffer. Total cost for those two devices: $6. The other 32 devices are NAND Flash chips. Total cost for those devices: $64 for 64 Gbytes of storage (not counting spare capacity). The cost of the NAND Flash devices is more than 90% of the silicon cost of an SSD. The SSD’s price is largely set by the cost of its internal NAND Flash.

That’s why SSDs aren’t likely to replace HDDs for bulk storage in the foreseeable future. As long as the HDD industry has a road map leading to higher capacity and lower cost/bit storage, and it does, then the HDD will keep the throne as the storage capacity king.

SSDs can beat HDDs in raw performance by one or two orders of magnitude, as measured in IOPS. There’s nothing on the HDD road map that can change that situation. For applications that can measure the value of storage speed, and there are many such applications for enterprise-class storage, SSDs provide sufficient value to justify their higher price/bit. For most consumers, people who are selecting laptops for example, the choice between a 160-Gbyte HDD or a 32-Gbyte SSD for the same price is obvious. The consumer will choose more capacity (to store more music, more pictures, more video, and more movies) every time.

Now take a look at Handy’s curves for DRAM and NAND Flash cost/bit once again:

Note that the cost/bit of NAND Flash is now roughly 10% that of DRAM. That means that as a DRAM backup medium, NAND Flash doesn’t add that much to the cost of the DRAM it’s backing up. Unlike the comparison of NAND Flash and HDD capacity, which tilts far in favor of the HDD, NAND Flash densities are much better than DRAM bit densities and that gap is growing thanks to multi-level cell (MLC) storage. These economics are behind the idea for AgigA Tech’s AGIGARAM modules. For a small cost adder, volatile DRAM can be made bulletproof when paired with NAND Flash memory. For more detail regarding this idea, see the earlier 3-part series in this blog (here, here, and here).

For this blog entry, we’re returning to the Flash Zone, a concept described by Denali Software’s CTO Mark Gogolewski in his keynote speech—The World is Flash: A Disruption of the Memory & Storage Hierarchy—at Memcon 2009. The Flash Zone is the name put to the performance gap between DRAM and disk storage. There’s not only a gap in performance within the Flash Zone, there’s a transition from volatile memory (DRAM) to non-volatile storage (hard disk). With steep cost/bit price declines and per-device capacity growth, NAND Flash devices now easily fit into this gap and produce a new and viable layer in the overall computer memory hierarchy.

What’s new is that Jim Handy’s keynote at the Bell Micro SSD seminar put some welcome numbers on the Flash Zone that further clarify Flash’s place in the hierarchy. Here’s an image of that particular slide.

This image plots the performance and cost of the different memory hierarchy layers from first-, second-, and third-level processor cache through DRAM, disk, and tape. Because Handy’s used a log-log scale to plot everything, the graph looks nice and linear even though the reality is quite a bit messier. For a conceptual graph however, this’ll do nicely.

Note that there’s a gap in the hierarchy. That’s the Flash Zone. Here’s the same plot augmented a bit. The big red circle identifies the Flash Zone.

Also note that Handy has labeled the gap and says it’s “growing.” The gap’s growing because DRAM is getting faster, bigger, and cheaper, moving its ellipse up and to the left while HDDs are getting bigger, although not much faster, moving the HDD ellipse horizontally to the left. The result is a growing performance and bandwidth gap between DRAM and HDDs.

Flash fits into this gap very, very nicely said Handy (and as discussed in this blog previously). Later in his keynote, he displayed this image to underscore the point.

There are currently at least three ways to fill the Flash Zone in a memory hierarchy using NAND Flash memory. The first way, the way that gets the most attention these days, is with solid-state drives (SSDs). Because they employ the same interfaces and share the same form factor with HDDs, SSDs are an easy, drop-in Flash Zone filler. They boost performance just by dropping them into place as HDD replacements, although that may not be the best way to introduce SSDs into the hierarchy. (More about that in a later blog.)

The second way to drop NAND Flash memory into the Flash Zone is through direct- or I/O-attached drives. This is the approach advocated by Fusion-io, as discussed in that earlier AgigA Tech blog entry on the Flash Zone. Direct-attached SSDs eliminate the HDD interface and protocols, which were designed with built-in assumptions about the performance characteristics and limitations of HDDs (“rotating rust” quipped Scott Stetzer, VP of Marketing at SSD vendor STEC). Free of those limiting assumptions and limits, direct-attached SSDs deliver more performance than do SSDs employing HDD interfaces.

Handy showed the ways to introduce these two types of SSDs with the following slide:

In enterprise-class server systems, SSDs with HDD interfaces typically plug into SAN racks and tie to servers over a network while direct-attached SSDs plug directly into the server over a high-speed interface (typically PCIe). Note that smaller servers with HDD interfaces often talk to SSDs directly.

Because he was speaking at an SSD seminar, Handy did not discuss the third way of introducing NAND Flash into the Flash Zone—the approach employed by AgigA Tech’s AGIGARAM. That approach mates the NAND Flash directly to the server’s DRAM, creating a high-bandwidth connection between the two memory hierarchies. In this application, however, the NAND Flash is used for DRAM backup and power-failure bulletproofing—not necessarily for storage (although there are other possibilities to be discussed in this respect).

So far, we’ve only been able to discuss two of Handy’s 47 keynote slides. The talk contained a ton of good information for server designers and enterprise system architects. More later.

Note: Handy’s keynote was based on his company’s new report: Solid State Drives in the Enterprise – 2010.

• Solid-state disk drives (SSDs). With SATA and SAS interfaces, SSDs can plug directly into most existing systems and will provide an immediate performance boost.
• Flash caches. Located very near the DRAM, Flash caches provide fast ways to back up DRAM data over wide memory buses with high bandwidth and low latency. AgigA Tech’s AGIGARAM and Denali’s FlashPoint controller are both aimed at this NAND subniche.
• Specialty storage devices based on non-disk interface standards. Disk interfaces including SATA and SAS were developed with built-in assumptions about the drives they support. Those assumptions include some temporal assumptions based on having rotating mechanical memory. Those assumptions don’t apply to NAND-based storage devices so it’s possible to use interfaces with more bandwidth, PCIe and Hypertransport for example, to connect such storage and get better performance. This is the sort of product available from Fusion-io.

Which finally brings us to the trigger for this blog entry. The MIT/Stanford Venture Lab (VLAB) held a panel discussion at Stanford University on Tuesday, November 17 and the evening’s topic was “SSDs: Game-Changing Technology for Better, Bigger, Faster Applications and Application Development” and the first speaker was David Flynn, President and CTO of Fusion-io. Flynn’s talk contained many interesting and worthwhile things for followers of NAND-related topics as they relate to computer system design.

Early in his presentation, Flynn projected a photo of Charlie Chaplin playing one of the last great roles of his life, “The Great Dictator.” However, Chaplin’s roundish face had been replaced with an inset photo of a hard-disk platter and the caption was: “Getting rid of nasty Disc-tators.” Flynn emphasized that Fusion-io’s PCIe-connected products are not solid-state disks and they deliver more performance than solid-state disks because they are connected to a data pipeline that delivers more performance than existing disk interfaces. They are I/O-memory devices that provide 10x the capacity of DRAM per dollar, 50x the capacity of DRAM per “module,” and 100x the capacity of DRAM per Watt. Using these metrics, Flynn is making it clear that he understands the figures of merit valued by his company’s prospects.

Flynn then compared NAND Flash memory to aircraft aluminum. When metallurgists developed aluminum alloys suitable for aircraft, the entire airframe had to be re-engineered because aeronautical engineers couldn’t use aluminum as a direct replacement for wooden struts and dope-covered fabric. Aluminum ushered in new types of airframes that rapidly evolved. Aircraft performance soared as a result.

The same is true of computer systems (and software) developed before and after the Flash Zone is filled with something, whether it’s SSDs, Flash caches, or I/O-attached storage. Assumptions must be rethought and systems and software need to be redesigned to fully exploit the advantages of a populated Flash Zone.

The question’s not as simple as it seems. There’s a temporal quality to the question. What’s right ten years ago isn’t right today and probably won’t be right ten years from now. Semiconductor technology is both fluid and extremely dynamic. One thing’s certain. You need to deal with today’s problems today. If you can address the same problem in the same way two or three years from now, that’s great! But you still need to address today’s problem today. You need to use components you can get today, not some time in the future. The future may include some surprises that change today’s answer, but today’s answer must be based on what you can do today.

Why the emphasis on today? Well, any RAID server memory used today must be based on some sort of memory technology (or technologies) that’s commercially viable now. Researchers are working on more than a dozen new memory technologies that may someday produce a more ideal memory than the semiconductor memories we have at our fingertips today. It’s not clear when that might happen. Tantalizing technology announcements are made almost weekly. But technology announcements are generally light years away from being commercially competitive products and that’s never truer than when you’re talking about digital memory.

Bulletproof RAID server memory must have some mechanism to ride through power outages without data loss.  The previous two entries in this series (Part 1 and Part 2) discussed various approaches to creating bulletproof memory using battery-backed RAM. Seems like a great idea, but batteries aren’t particularly reliable in data-center environments where they live inside of heat-generating boxes squeezed into rack upon rack upon rack where they get no light and precious little maintenance. High-maintenance components like batteries just seem like a poor choice for creating memory that’s supposed to be bulletproof. Wouldn’t you agree?

So what’s that leave?

Well, you could use NAND Flash for memory rather than DRAM. NAND Flash devices have many excellent attributes. They do not require power to provide nonvolatile storage. They are currently the semiconductor industry’s cost-per-bit leader. NAND Flash chips available in higher capacities than DRAMs, which translates into more bits per same-size board, fewer devices per board for same-size capacity, or smaller boards depending on application needs. These are all great attributes.

Unfortunately, NAND Flash devices have some unhappy qualities as well. You can only write to them relatively slowly—much more slowly than DRAM. They also exhibit wearout failure, which is getting to be a bigger and bigger problem as lithographies shrink. NAND Flash devices are block oriented so you can’t write just one word. These three failings are major and make NAND Flash memories unsuitable for RAID server memories.

Unsuitable, that is, when used alone.

However, volatile DRAM paired with non-volatile NAND Flash make a pretty good team when it comes to building bulletproof RAID server memory. When the power’s good, use the DRAM like…well…DRAM. When there’s an indication that power’s about to fail, save the contents of the DRAM in NAND Flash devices.

Note that you can’t let the host CPU save the data when power’s already on the slippery downhill slope. You really don’t know how much time there is before the host CPU loses its mind. You need something more—bulletproof. You need a backup power supply that will sustain the memory subsystem during the data-backup operation and you need a local processor to oversee the transfer.

Batteries are still bad

The previous two installments of this series have already dealt with the many reasons that batteries are not suitable as the backup power supply. Barring the sudden invention of the Mr. Fusion portable reactor last seen attached to the back of Doc Brown’s DeLorean time machine in the Back to the Future movies, there’s really only one good alternative for emergency backup power for RAID server memories: ultra-capacitors.

Ultra-capacitors are capacitors that have electrodes with greatly expanded area, which result in greatly expanded capacitance. The electrode area expansion originates in porous carbon electrodes. Ultra-capacitors have capacities measured in Farads, much greater then conventional electrolytic capacitors. Although they require the proper care when designed into a backup power supply, ultra-capacitors can provide enough backup energy to support the emergency transfer of data from DRAM to NAND Flash memory in a bulletproof RAID server memory subsystem.

How practical is all this? Very practical. Take a look at the following graph, which plots projected memory costs in dollars per megabyte over the next few years. (This graph is based on iSuppli projections.)

As you can see, DRAM and NAND Flash are the least expensive semiconductor memories, per megabyte, and a megabyte of NAND Flash costs about one tenth of what a megabyte of DRAM costs. All of the leading “future” memories, which may someday replace DRAM, cost more. Some cost much more and they will continue to cost more into the foreseeable future. These “future” memory technologies are not about to replace DRAM today or tomorrow. They cost too much.

Finally note the dashed blue line. This line represents the per-bit cost of AGIGARAM, which fuses DRAM, NAND Flash, and ultra-capacitors to create the closest thing to a bulletproof RAID server memory that you can get today. Over time, the cost of a megabyte of AGIGARAM approaches the cost of the equivalent amounts of DRAM and NAND Flash added together. The cost of the memories will essentially dominate the other costs (controller, ultra-capacitor backup power source). Consequently, AGIGARAM, which is AgigA Tech’s bulletproof memory for RAID servers that’s available today, is not only the best technical approach to creating bulletproof memory, it’s the most cost-effective approach available today…and tomorrow.

As we discussed in Part 1 of this blog entry series, RAID servers must use non-volatile memory for their write caches to prevent data loss during power failures. There are many ways to achieve nonvolatility. One way is to back up the entire server with an uninterruptible power supply. That takes a lot of battery power or a diesel-driven generator (or a hydroelectric turbine, if there’s one handy). Another way is to use a much smaller battery to back up the RAM used as a write cache. Yet another is to use NAND Flash as a write cache. All of these design approaches have problems and no matter the approach, the server processor must be involved in safely preparing for the imminent loss of power. Let’s examine these last two design approaches more closely, assuming that diesel generators and water power are out of the question.

Backup batteries have short lives and require regular maintenance, which they often do not get. NAND Flash memory has relatively slow write times, so it makes a poor write cache when used directly. Worse, NAND Flash memory exhibits write-induced wearout failure. You really must minimize the number of times you write to Flash memory. For both of these reasons, using Flash memory like it’s RAM is clearly a misapplication of Flash memory technology.

So what’s the real cost?

Back to the original question posed in this blog entry: What’s the real cost of the memory in a RAID server? Let’s run a thought experiment and see where it takes us. Consider a battery-backed RAM. Besides the cost of the RAM, which is the same whether there’s battery backup or not, there’s the cost of the battery. What’s the cost of a battery pack? It’s on the order of $100 for the RAID server customer. However, if your customers are replacing these batteries annually as they should, then there’s roughly $500 worth of batteries to buy per server over the course of a four-year lifespan for the memory. (That’s $100 initially for the first battery and $100 per year for each year following.)

However, that’s not the only cost. Someone must go into the server room, take the server down, replace the battery, and then bring the server back up. For the sake of argument, let’s say it takes an hour for an IT tech to do all of this for one server. What’s the burdened cost of an hour of an IT tech’s time? Well, that number varies, but again it’s on the order of $100. And you need to do it four times over the course of the 4-year life of the server memory. That’s another $400. (We’re ignoring recycling costs here, but batteries should be recycled properly.)

So if battery maintenance occurs as it should, the cost of non-volatile server memory is roughly the cost of the memory plus $900 in maintenance costs. These costs greatly exceed the cost of the memory itself.

But what if battery maintenance doesn’t take place as it should? What if the battery fails in service? What’s the cost then? Well, in this scenario, you need to make some big assumptions. First, you need to assume that the batteries are all properly monitored so that there’s an alert as soon as a battery fails. If not, then the RAID servers are always subject to catastrophic data loss because their write caches are unprotected from power failures. Actually, it’s not so easy to sense battery failure without putting a load on the battery, but let’s ignore this detail for now.

Next you need to assume that there’s a replacement battery handy, sitting ready to go on the shelf next to the server room, and that someone knows where this battery is stored. Otherwise the RAID server with the failed battery will need to be taken out of service and replaced with another server until a new battery can be found, flown in, or otherwise delivered from the warehouse, wherever that is. Battery spares are cheaper to keep on the shelf than spare RAID servers so it’s likely that it’ll be a spare battery on the shelf. Likely as not, the battery on the shelf won’t be fully charged, but let’s ignore that detail for now as well.

Finally, you need to assume that there’s always an IT tech on hand who knows how to replace a server backup battery and can act quickly when a battery fails.

These are all big assumptions and they are all most assuredly bad assumptions, but they set a lower bound on the associated maintenance costs. An unattainable lower bound, most certainly, but a lower bound nevertheless.

$300 for one failure, $500 for two

If you make all of these assumptions, then the costs for server-memory nonvolatility using battery backup include the initial $100 battery cost, plus the cost of replacing the failed batteries over the four-year life of the server memory. In the highly unlikely event that there’s only one failure during that time, the 1-time replacement cost is about $200 ($100 for the replacement battery plus $100 for the labor cost to replace it) for a total of $300 for the initial battery plus one replacement. If the battery fails twice during the four years, then the total cost is $500.

While this second scenario sets a lower bound on cost, it’s clearly built on unrealistic assumptions. There will most certainly be unplanned downtime with this scenario.

Batteries almost never fail at convenient times. They seem to have a second sense about these things. Batteries fail at night and when the IT team is otherwise occupied. So you also need to figure in the cost of lost business due to the unplanned server outage. Realistically, that’s clearly going to happen.

Lost time counts too

Now the dollar value of lost data is really tough to set. However, as discussed in the previous blog entry, an hour’s loss of server time could easily cost a large customer thousands or millions of dollars especially if that server customer is Amazon, Google, or a fast-transaction securities trader that relies on response times that are microseconds faster than competing traders. For such customers, the cost of server memory is clearly irrelevant because uninterrupted server uptime is so very valuable to them. These customers know to the penny what server uptime is worth per minute, per second, and even per millisecond. That’s how valuable server uptime is to this class of customer.

These customers don’t want to know how much the memory in the server costs. They want to know how the server’s design will prevent unplanned downtime.

The server design team must therefore have bulletproof, nonvolatile memory as a goal. This memory should not require annual maintenance so that the server’s design avoids both frequently planned and unplanned downtime due to memory failure. The economics of this goal are simply undeniable.

If you’re thinking that this discussion is leading to a discussion of why AgigA Tech’s approach to non-volatile server memory is worth more money, you’re wrong. After taking maintenance costs into account, AgigA Tech’s AGIGARAM modules actually cost less. Taking the cost of lost data and server downtime into account, AGIGARAM modules cost a lot less. Something to be discussed in the next blog entry.

In fact, one of those batteries has failed. The RAM cache it protects is at risk when the next power outage occurs. When that happens, one or more of the data center’s customers will lose data. Critical data. After all, what data isn’t critical?

Worse, the failed battery is leaking. Acid is oozing out of the battery. It’s quite possible that the acid is leaking onto critical circuitry inside of the RAID enclosure. Drip. Drip. Drip. The acid starts to etch into the circuitry. The disaster is perhaps moments away…

Customers buy one thing from RAID vendors: a safe haven for their bits. The bulletproof aspect of a RAID array’s disk storage resides in the redundancy of the disk drives themselves. A RAID 5 array protects against data loss should one disk drive fail and a RAID 6 array protects against faults should two drives fail. Both types employ disk striping with parity (double parity for RAID 6). Because data has value—and some data has tremendous value—the use of RAID systems based on hardware RAID controllers is skyrocketing. However, power loss can negate the efficacy of a RAID system and puts the data at risk.

One critical point of failure in RAID systems with respect to power outages is the write cache. RAID systems employ write caches to speed disk transactions—to boost the IOPS (I/O operations per second) rating. Once a computer system squirts a chunk of data into a RAM-based cache, the RAID system can immediately acknowledge the transaction before actually writing the data to disk. So there’s a critical period of time when the data is at risk from a power failure, after the acknowledgement but before the data is on the disk. If power is lost while the data is in RAM cache, then it’s lost forever.

One way to avoid this problem entirely is to disable the RAID system’s RAM cache. This approach preserves the data but with a huge performance hit. No RAM cache, no performance.

Another way to avoid the problem is to protect the data in a write cache from power failures using a battery-backup unit (BBU). That way, the RAID controller can recognize an impending power failure, can halt transactions, and the BBU will maintain any data yet to be written to disk and thus ride through the power failure.

Sounds great in theory, but in practice there are many problems with BBUs:

• Batteries have short, finite lifetimes compared to other electronic components and heat further shortens their electrochemical lives. There’s heat aplenty inside most server enclosures. Consequently, battery health should be closely monitored but it’s often not monitored at all. In fact, some data-center operations teams are surprised to discover that there’s a high-maintenance battery inside of many RAID systems. Of course, by the time they realize that there’s a battery to be maintained, it’s often too late because the event that brought this fact to light was a data failure induced by power loss.

• Batteries need to be replaced every one to two years. First, that’s not going to happen if no one knows there’s a battery to be replaced. Second, battery maintenance often falls pretty low on the priority list of tasks to be performed and the replacement may be dangerously deferred when it’s done at all. Third, there’s no standardization in BBUs so the correct battery pack may not be on hand. Worse, the required BBU may be discontinued, no longer be available. If you can’t order a new one, then what? Fourth, battery packs cost money and so does the time it takes to install new ones.

• When replacing the BBU, the RAID server must be taken offline, or at least the RAM cache needs to be taken off line and it must stay off line until the BBU charges up. RAID performance suffers during the downtime. Consumer-level products such as PCs and PVRs (personal video recorders) may not benefit much from faster disk drives. Enterprise systems do. Enterprise computing clients know precisely what a second’s worth of delay costs in their business. Sometimes a microsecond’s delay costs big money. For example, Google and Amazon know to the penny what each additional second of response delay costs them in terms of lost customer purchases. High-frequency securities traders and arbitrage houses employ trading strategies that are highly dependent on ultra-low latency networks. In fact, they co-locate their trading servers with the trading floor to minimize communications latency with the computers at the market exchange. These traders profit only by feeding information on competing bids and offers to their trading algorithms microseconds faster than their competitors. Loss of write-cache performance in a RAID system could literally cost such traders millions of dollars per microsecond of delay.

• Batteries are not environmentally friendly so it’s a bad idea to just toss them in the trash. Batteries should be properly recycled and proper recycling is expensive, beyond the cost of the replacement BBU. Even when recycled properly, batteries just aren’t that great for the environment.

So what’s the right answer to the need for bulletproof RAID write cache? AgigA Tech believes that the answer can be found in a fusion of NAND Flash and ultra-capacitor technologies. Ultra-capacitors are essentially made of benign carbon and have many superior qualities compared to batteries. In particular, they charge faster (less downtime) and they have longer lives (when properly applied). NAND Flash can save a RAM cache’s contents indefinitely and without power. So AgigA Tech’s AGIGARAM modules can be used as RAID RAM-cache modules, providing all of the benefits of battery-backed write caches but without the many liabilities batteries incur.

What about the cost of such an approach? Stay tuned. We’ll address that in the next blog entry.

PCM can’t perform the same trick of packing multiple bits per cell that Flash does by using variable amounts of charge to represent two, three, or four bits on one cell. One of the known and required enabling technologies for PCM to become cost-competitive with Flash memory is the ability to physically stack multiple bit cells to create 3D PCM devices. That’s precisely what researchers from Intel and Numonyx announced last week. Researchers from the two companies—who have been working on PCM together long before Numonyx spun out as a in independent company in 2008 after starting as a joint partnership between Intel and STMicrolectronics—said they will be presenting a paper at next month’s IEDM conference on a test chip that implements stackable (but not yet stacked) PCM cells.

The stackable PCM cell consists of a chalcogenide memory cell and another chalcogenide device called an Ovonic Threshold Switch (OTS), which was developed by Stanford Ovshinsky more than 40 years ago. Ovshinsky is the all-time, grandmaster alchemist of silicon. His specialty is the study of amorphous silicon, essentially glass, and he has found an incredible number of uses for this material. Both elements in the PCM memory cell announced by Intel and Numonyx owe their existence to Ovshinsky.

The OTS is a 2-terminal breakover device. Insufficiently excited, an OTS doesn’t conduct electricity. With a large enough electric field across it, the OTS quickly becomes a conductor. Reduce the field below the threshold and it becomes nonconducting again. It’s a diode-like device made without a semiconductor junction. In addition, an OTS is made of similar stuff to the PCM memory cell itself: chalcogenide glass. Consequently, an OTS makes a great cell selector for a PCM bit cell. The Intel/Numonyx test chip sandwiches a PCM memory cell and an OTS between a row and column line to create a stackable PCM memory cell. Energize the row and column lines sufficiently and current will flow through the OTS. That’s all that’s needed to create a stackable PCM cell, according to the announcement.

AgigA Tech specializes in fusing fast DRAM with non-volatile semiconductor memory to create bulletproof memory subsystems for embedded systems and servers. Currently, the non-volatile semiconductor memory of choice is Flash EEPROM because of the tremendously advantageous cost/bit, data-retention time, and raw bit-storage capacity. However, Flash may not always be king of the non-volatile hill and so it’s important to stay on top of non-volatile memory developments such as the one announced last week by Intel and Numonyx.

There’s a similar but even larger performance gap between the access time of DRAM and HDDs. Although vendors have improved HDD capacity by 60% per year—each and every year—and HDD’s price per storage bit directly tracks that trend as well, there’s been very little improvement in HDD data transfer rate and interface speed and there’s been no dramatic change in HDD access time, which is largely determined by mechanical factors. Consequently, there’s been only a relatively slow improvement in HDD IOPS (I/O operations per second), which leads to a massive five-orders-of-magnitude (10^5) performance gap between DRAM access times and HDD access times and that performance gap is growing.

At the same time, DRAM’s volatility plays a role in a system’s sensitivity to power glitches and losses. When data is critical, and most data is critical these days, non-volatile memory just isn’t sufficient. Some means of retaining data through a power loss is usually required. In the past, HDDs have sufficed for non-volatile storage but they’re simply too slow these days.

The Flash Zone

Flash memory is a good candidate for filling this memory gap because it provides nonvolatile storage and it has become the cost-per-bit leader in semiconductor memory. Consequently, Mark Gogolewski, Denali Software’s CTO, calls this performance gap in the memory hierarchy the “Flash Zone” (see figure below and the Reference), the performance zone between DRAM and HDD access times.

The Flash Zone in memory hierarchy

The reasons for Flash memory’s candidacy to fill this gap include:

• In 2004, the per-bit cost of NAND Flash dropped below the previous category leader, DRAM.
• More NAND Flash bits shipped in 2005 than bits of any other type of semiconductor memory.
• More NAND Flash bits shipped in 2007 alone than all of the DRAM bits shipped in the last 25 years of commercial DRAM production.

There’s been a huge decrease in the per-bit cost of NAND Flash and a big capacity increase on a NAND Flash die. Consequently, NAND Flash memory fits nicely in the gap between DRAM and HDD. It offers faster access speeds than HDDs by at least two orders of magnitude while replicating an HDD’s non-volatile storage abilities. In addition, NAND Flash memory can draw considerably less power than HDDs when managed correctly. The opportunity for innovation in memory hierarchy is therefore huge.

Three Ways to Use NAND Flash: SSDs, Flash Cache, DRAM Backup

There are three ways to fill the Flash zone. The first approach is to use NAND Flash memory to create an HDD emulator using the same disk interface and possibly even the same form factor. Such drives are called solid-state drives (SSDs) and they have been gaining traction in the industry. Because they do not employ rotating memory, SSDs can deliver far faster access times than HDDs. However, there are costs associated with this approach. SSDs cost substantially more per stored bit than HDDs while retaining the overhead associated with HDD interfaces and protocols. The memory bus protocols and interfaces used to connect DRAMs to processors are much, much faster.

At this time, most analysts agree that NAND Flash memory will not overtake HDDs in cost per bit. Jim Handy of Objective Design presented the chart shown below at MemCon 2008 showing that the 25x cost-per-bit advantage for HDDs relative to NAND Flash memory cost per bit would continue for the foreseeable future.

NAND Flash and HDD cost-per-bit forecasts
(Jim Handy, Objective Design)

Denali’s memory market analyst Lane Mason recently commented that the pace of cost-per-bit reductions for NAND Flash memory will actually slow compared to price drops in recent years. So it doesn’t appear that NAND Flash will supplant HDD storage in the near- or medium-term future.

The second way to use NAND Flash memory in the Flash Zone is called a Flash cache. A Flash cache speeds access to an HDD by buffering the data stream between a processor and the HDD. Data is drawn from and written to HDDs as needed and the same data is simultaneously cached in NAND Flash. The next time this data is needed, it’s drawn directly from the Flash cache instead of the slower HDD. Flash caches do not require as much NAND Flash memory as SSDs, and therefore cost less, but they can deliver performance improvements when paired with HDDs.

The third way to use NAND Flash memory is to implement a backup strategy that allows the DRAM to operate normally when system power is available and to quickly save that data in non-volatile NAND Flash when system power fails. In this approach, which is used in AgigA Tech’s AGIGARAM Non Volatile System (NVS) modules, a backup power source provides the energy needed to safely tuck data away in non-volatile storage (NAND Flash), which then retains the data for a decade or more if needed.

This third approach to filling the Flash Zone offers several benefits including:

1. Fast backup when power fails
2. No energy required to save the data during power failure
3. Automated backup and restoration of data with no host-based software assist required

Which of these three approaches to use depends on the application (as always). If you’d like help deciding, please feel free to contact AgigA Tech.

Reference

The World is Flash: A Disruption of the Memory & Storage Hierarchy, Keynote Speech, Denali Memcon 09, Mark Gogolewski, CTO, Denali Software, Inc., www.denali.com

Of course, the culprit is you. You didn’t properly maintain the device by instituting a program of regular maintenance and periodic battery checkups. How could you? The device was tucked away on a shelf, perhaps years ago, and you forgot all about it. After all, batteries aren’t the most important thing in your life. You’re a busy person.

This same scenario applies to batteries in all sorts of embedded equipment and servers. As little electrochemical electricity factories, batteries need to be maintained or they will eventually give you a nasty surprise. It’s that simple.

Over the last 10 years or so, ultra-capacitors have started to replace batteries in a variety of electronic applications where large amounts of energy storage are needed. One of the primary uses for ultra-capacitors is memory backup. A more conventional approach to memory backup employs ultra-capacitors to provide backup power to SRAM or DRAM subsystems in the event of a power outage. However, even large banks of ultra-capacitors cannot back up memory for years. A different sort of approach to preserving data in the event of a power mains failure is to draw energy from ultra-capacitors just long enough to move critical data from volatile SRAM and DRAM into non-volatile Flash memory. Then the Flash memory can retain the data for ten years or more with no power at all.

Ultra-capacitors get their high capacities from porous carbon electrodes that provide massive amounts of surface area in tiny spaces. As nanotech research delves into the mysteries of carbon-based nanostructures such as nanotubes, ultra-capacitor storage capacities improve. Coincidentally, the number of ultra-capacitor vendors has recently been increasing and therefore the effort required to evaluate the various offerings has also been increasing.

Characterizing an ultra-capacitor isn’t simple. For long-term use in critical embedded and server systems, you need to know how an ultra-capacitor’s electrical storage abilities change over time, temperature, and voltage. It turns out that the long-term characteristics of these low-voltage devices are extremely sensitive to temperature and to operating voltage. They’re also sensitive to the way they’re charged and discharged, so the design of charging and discharging circuitry is critical to the safe, long-term use of ultra-capacitors in multiple-device banks.

If you’re designing memory subsystems and wish to use ultra-capacitors for power backup, you have two choices. On the one hand, you can mount your own ultra-capacitor characterization program and develop your own charging and discharging circuitry. Alternatively, you might choose to use a pre-designed, pre-characterized power module based on ultra-capacitors that’s specifically designed for memory-backup applications such as the PowerGEM offered by AgigA Tech. Either way, ultra-capacitors offer a good alternative to backup batteries, one well worth investigating.

References:

Energy Storage Industry Needs Novel Circuits And Semiconductors, Bobby Maher, http://electronicdesign.com/Articles/Index.cfm?AD=1&ArticleID=21973&bypass=1

Ultracapacitors Challenge the Battery, John M Miller, http://www.kilofarad.org/files/Ultracap-%20World%20&%20I%20-%20June%202004.pdf

1. October 10, 2009. T-Mobile Sidekick owners found that they’d lost their contacts, calendar entries, to-do lists and photos when Microsoft subsidiary Danger suffered a technical glitch. At first, the news was very bad. The lost data looked unrecoverable. Then, it looked like some of the data might be recovered. Then most. If you are or were a T-Mobile Sidekick user, what would you be thinking about the service right now?

2. September 24, 2009. Google’s Gmail blows up, again. Only a “few” users are affected, but it’s the fourth time in two years that Gmail has made the news because of service loss.

3. September 6, 2009. Twitter fails for hours. Sure the Twitter Fail Whale shows up regularly, but Twitter is a high flyer with huge visibility.

4. August 3, 2009. eBay’s PayPal crashes for five hours. PayPal loses millions of dollars in transactions that don’t happen. PayPal’s merchant customers lose more.

5. June 29, 2009. Rackspace loses power in its Dallas data center and ends up rebating customers millions of dollars in usage credits for lost service.

6. January 6, 2009. Salesforce.com’s servers crash for about half an hour. One blogger notes: “Salesforce demonstrates how (un)reliable SaaS really is.”

This sort of press is a server provider’s worst nightmare. One of the missions of this blog will be to propose approaches to improving server reliability. Please feel free to contribute your ideas.