2x nm will happen – Currently, vendors are shipping 34nm NAND Flash memory in volume. That’s a tremendous engineering feat in itself. As reported in an earlier blog entry (“Intel, Micron striving to regain lead in NAND tech”), Intel and Micron Technology are apparently planning to ship NAND Flash devices based on 2x nm lithography (called 2x because it’s not yet clear if it’s 26nm, 25nm, 24nm, or 22nm). Lithography shrinks are a true race to the bottom. Features on chips have become so small that one or two atoms of difference from one part of a chip to another cause real changes in device characteristics. This phenomenon is called on-chip variation or on-chip variability (OCV) and it’s a tough problem to tackle, requiring the use of smarter on-chip circuitry to deal with the variation. (See “My Head Hurts, My Timing Stinks, and I Don’t Love On-Chip Variation” by Matt Weber.) The problems do not appear to be insurmountable and NAND Flash vendors currently leading the lithography derby appear intent on keeping that lead until it’s no longer possible.

SDDs will get bigger and faster – This prediction needs to be written, but it’s really a no-brainer. The “bigger” part of the prediction is based on the ever-increasing capacity of the NAND Flash chips used to make SSDs. They will get bigger, driven by economic forces far beyond SSD usage. USB Flash-memory drives are the big volume driver in this market and there’s always demand for more capacity there while the form factor emphasizes small and slim. Nowhere to go but more on-chip capacity. Number two driver is SD cards for cameras and camcorders, with the same forces at work. As for faster, it’s clear that SATA 6G is in the immediate future for SSDs. Marvel’s SATA 6G controller (see “Early Results Show SATA 6G Performance All Over the Map”) and Micron’s introduction this month of an SSD that exploits SATA 6G to good effect (see “New SSD Introductions from Seagate and Micron”) clearly point the way to faster SSD operation, once the internal architectural designs are aligned with the faster interface.

ONFi 2.1 gets big – ONFi stands for the “Open NAND Flash interface” and the ONFi organization (www.onfi.org) bills it as the fastest Flash interface on the planet. Hyperbole aside, ONFi supports transfer rates to 200 Mbytes/sec. and that’s speedy in anyone’s book. The ONFi Working Group was formed in May 2006 and currently has over 80 member companies including Hynix Semiconductor, Intel Corporation, Micron Technology, Inc., Numonyx, Phison Electronics Corporation, Sony Corporation and Spansion. Wanna fight those guys as a group?

MLC and TLC get big – It’s already happening. Several NAND Flash vendors already offer MLC (multi-level cell) and TLC (three-level cell) NAND Flash devices. They are working to improve the reliability of these devices while SSD and other subsystem manufacturers are working to develop system-level techniques to mask the reliability of these devices. That’s not a patch job. HDD vendors have spent the last 50 years paving over the data-reliability problems of rotating magnetic storage and no one pays much attention any more except the engineers tasked with keeping those problems at bay. Much the same will happen for NAND Flash devices and for products based on those devices.

NAND Flash prices firm up – NAND Flash prices have recently risen and a lack of capital investment in new fabs and processing equipment foretells the usual period of spot shortages and price peaks associated with scarcity. See MemoTrek’s extensive analysis: NAND Flash Prices: 4Q Trends & 2010 Forecast.

Joker’s wild – If we told you, then it wouldn’t be a surprise, would it?

Please have a happy and safe New Year’s celebration and we’ll see you next year.

Seagate announced its Pulsar SSD line on December 7 or 8 (depending on which version of the press release Google finds for you), allowing a show to drop that people had expected for more than a year. Pulsar drives use the familiar 2.5-inch HDD form factor and a SATA interface, making it easy to drop the drives into existing computer and server systems. Seagate’s Pulsar SSDs employ SLC (single-level cell) NAND Flash devices, which cost more per bit than MLC (multi-level cell) and TLC (three-level cell) NAND Flash devices. In exchange for the higher cost, you get more reliable memory, as was discussed in this blog a while back. (Check out “More than Moore: SLC, MLC, and TLC NAND Flash.”)

Seagate Pulsar SSD

The use of SLC NAND Flash underscores Seagate’s focus on enterprise-class storage for the SSD. There are at least two good reasons for Seagate’s enterprise focus. First, enterprise customers are more able to translate an SSD’s speed advantage over HDDs into dollars (as previously discussed in the blog entry “SSD TCO (Total Cost of Ownership”). Second, SSDs are a premium product with a premium price. Enterprise customers more easily accept the higher cost/Gbyte price tag attached to SSDs. Seagate’s Pulsar SSDs are available in storage capacities to 200 Gbytes and the SSDs achieve “a peak performance of up to 30,000 read IOPS and 25,000 write IOPS, 240MB/s sequential read and 200 MB/s sequential write” according to Seagate’s press release. The Pulsar drives have a 5-year limited warranty.

Micron Technology rolled out its RealSSD C300 less than a week before Seagate’s SSD announcement. The first glaringly obvious difference in Micron’s C300 SSD is that it sports a 6-Gbyte/sec SATA 6.0 interface. However, the faster interface alone will not boost performance (discussed earlier in this blog here) if the drive internals aren’t designed to sustain high transfer rates supported by SATA 6.0. To that end, Micron’s RealSSD C300 press release discloses the fact that the new Micron SSD “leverages a finely tuned architecture and high-speed ONFI 2.1 NAND Flash  to provide a whole new level of performance.” (ONFi, the Open NAND Flash interface, is discussed in this previous blog entry.) The result: a read throughput speed of up to 355MB/s and a write throughput speed of up to 215MB/s.

Compare those numbers to Seagate’s Pulsar and you’ll see that the Micron drive’s read throughput is nearly 50% faster but the write throughput is only 7.5% faster. Write throughput is one of the Achilles’ heels of SSDs. NAND Flash devices had an erase/write cycle that simply takes time.

Micron RealSSD C300

Micron’s C300 SSDs will be offered in 1.8-inch and 2.5-inch form factors, with both form factors supporting 128- and 256-Gbyte capacities. Micron is currently sampling the C300 SSD in limited quantities and expects to enter production in the first quarter of calendar 2010.

Both companies are making smart moves into the SSD market. Seagate, like Western Digital and its acquisition of SSD vendor SiliconSystems in March of this year, recognizes that it’s not in the HDD business—it’s in the storage business and SSD storage is hot right now. Micron, like Intel, sees SSDs as a value-added way to package and market it’s NAND Flash devices. Both companies have made very smart moves into the SSD market.

Find those last two sentences confusing or contradictory? Here’s the explanation.

SSDs have no read/write heads or positioning arms. Instead, they consist of several NAND Flash chips and a controller chip. There’s an array of memory blocks on each NAND Flash chip. The size of the Flash memory block is the smallest amount of memory a NAND Flash chip can write in one operation because NAND Flash memory blocks are atomic with respect to erasure. You can’t write just one byte or word because you must erase the entire block before writing to the block. That means an SSD can only write an entire NAND block at a time.

Here’s a graphic from Jim Handy’s SSD keynote at the Bell Micro SSD seminar held in early December that helps to explain the situation:

Each SSD consists of a stack of visible NAND memory blocks that the SSD controller uses to store written data. There’s also a shorter stack of spare NAND memory blocks that hold data in temporary storage. These spare blocks are also used to replace a visible block when it wears out from repeated write/erase cycles. All NAND blocks are equally accessible, so there’s no time penalty for writing NAND blocks out of sequence as there is when writing on non-adjacent or non-contiguous tracks with HDD storage.

However, most virtual operating systems don’t write in blocks, they write in 4-Kbyte pages that are much smaller than NAND Flash blocks. For example, Numonyx’ 1-to-16-Gbit NAND Flash devices have 128-Kbyte blocks. As a result, modifying one 4-Kbyte page in a NAND Flash block requires a relatively complex sequence:

1. Read the data for the entire block from NAND Flash into a RAM buffer
2. Modify the appropriate page in the block image now stored in RAM
3. Write the block back to an erased NAND Flash block
4. Fix pointers to the new memory block
5. Erase the old memory block as a background task

Consequently, SSD performance varies over time and the performance varies depending on how many erased and spare memory blocks are available across all of the NAND Flash chips in the SSD. SSD performance also depends on the ratio of reads versus writes—because reads occur ten times faster than writes for SSDS—and they vary over time as the NAND Flash chips fill up.

The following figure from Handy’s keynote shows a 3D data surface plot representing the IOPS performance of one SSD. (The figure is from a presentation at the August 2009 Flash Memory Summit made by Esther Spanjer, Director of SSD Marketing at Smart Modular Technologies.)

The X axis of the surface shows the ratio of reads to writes and varies from 100% writes on the left to 100% reads on the right. The Y axis shows SSD performance in IOPS. The Z axis plots “block” size, from the SSD-level perspective (which is page size from the NAND Flash chip’s perspective, yes that’s confusing).

The first thing to note from this surface plot is that performance is a lot better on the right-hand side, which is dominated by reads. You’d expect that because SSD read performance is 10x better than SSD write performance. It’s the nature of NAND Flash memory. Note how fast the performance falls off as the percentage of write transactions increases. Then note that there’s a sort of saddle effect along the Z axis. The saddle peaks at 4-Kbyte blocks. Most SSD designs are optimized for 4-Kbyte blocks because most virtual operating systems employ 4-Kbyte blocks (and have for decades, in spite of the radical, orders-of-magnitude increase in memory use by both operating systems and application software).

So, clearly, when an SSD vendor gives an IOPS rating for an SSD, you need to take that one number with a grain of salt. SSD performance varies significantly depending on the read/write mix and on block size. Consequently, SSD performance can’t be captured in one or two numbers.

Next, Handy presented this graphic from SandForce (which makes SSD controller chips):

This graph shows an initial conditioning period during which the test preconditions (fills up) the SSD using sequential 128-Kbyte writes. The initial transfer performance (about 80 Mbytes/sec for the particular drive being tested) drops slightly as the drive fills and the internal SSD controller starts shuffling full NAND Flash blocks off to spare memory. The falloff isn’t big because the sequential writes place a predictable load on the SSD controller. However, when the test switches to random 4-Kbyte writes about 4000 seconds into the test, performance drops significantly because the SSD controller suddenly needs to make small changes to memory stored in the NAND Flash blocks but the drive’s full and there are no empty blocks. Blocks must be erased to make room for the new data and block erasure takes time. Consequently, there’s a big performance falloff as the controller starts to shuffle data around inside of the drive to make room for new data.

Perhaps more interesting is what happens when the test switches back to large sequential writes about 11,000 seconds into the test. Initially, the sequential writes cause the drive performance to vary wildly because the preceding random writes have scattered the spare blocks and left them distributed throughout the SSD’s internal NAND Flash memory space. Eventually, the SSD’s internal controller gets things sorted out and the performance for large sequential writes returns to the initial steady-state level.

(Note: This graph is not supposed to typify the performance of all SSDs. The graph shows the results of a test on one particular SSD.)

So what’s to be learned from all of this data? SSD performance measurement isn’t simple. Creating controllers and firmware that deliver optimum SSD performance isn’t simple either. As drive and chip vendors learn more about the use of NAND Flash for storage, they develop better algorithms for extracting more performance from the NAND Flash chips.

NAND Flash chips are complicated, whether used in SSDs or for server memory backup as with AgigA Tech’s AGIGARAM modules. It takes experience to get the most performance from these memory devices.

My thanks to Jim Handy for all of the great information in his Bell Micro keynote, and for generously letting me use the information in this series of blog entries.

With IC lithographies approaching atomic limits—and our absolute inability to pattern transistors using fractional atoms that inevitably leads to the subsequent slowing of Moore’s-Law scaling—a third dimension starts looking mighty attractive. Just as cities started to build up towards the sky to fit more people and more businesses into limited downtown real estate, IC designers would dearly love to find easy ways to pack more transistors, more gates, and more bits into the same limited on-die real estate. One way to do this is to build circuits in layers. Intel and Numonyx will be discussing a new way to build nonvolatile phase-change memory (PCM) ICs using multiple layers in a few days at the IEDM conference in Baltimore, Maryland. But NAND Flash designers have already discovered another way to pack more bits into the same space by stuffing existing 2D memory cells with multiple bits. This approach also represents a way to circumvent 2D limits—to put NAND Flash bit capacity on a trajectory that is “more than Moore.”

Flash memory stores bits as charge trapped in a transistor’s floating gate, which is an isolated island of semiconductor surrounded by insulator. Electron tunneling drives the charge into the floating gate through where it *mostly* remains trapped until erased. (Sometimes, the electrons wander off by themselves or through a phenomenon called “read disturb.”) The electrons trapped in the floating FET gate act like a phantom negative voltage that prevents the transistor from conducting when read. This is the original mechanism developed for the NAND Flash memory by Dr. Fujio Masuoka while working for Toshiba circa 1980. It’s a simple binary use of trapped charge. When charge is trapped in the NAND Flash cell’s floating gate, the associated transistor will not conduct when read. When there are no trapped electrons, the transistor will conduct during a read operation. You get a simple binary response to the trapped charge or lack of trapped charge.

However, there’s an essential analog mechanism available here. The Flash memory can trap more or fewer electrons in the floating gate. The variable amount of charge can be measured by a fast A/D converter. If you store four different charge amounts on a floating gate (empty, quarter full, half full, three quarters full, and full) then you have essentially put two bits (four states) worth of information in one NAND Flash memory cell. If you can trap and measure eight charge levels of charge in a NAND Flash memory cell, then you have essentially put three bits worth of information into one NAND Flash memory cell. NAND Flash memories that store one bit/cell are called single-level cell (SLC) memory. Store two bits/cell and you have multi-level cell (MLC) memory. Store three bits per cell and you have three-level cell (TLC) memory (or 3BPC—three-bit/cell—memory using Micron Technology’s terminology).

Great! Why not pack two or three bits worth of information into every NAND Flash memory cell and essentially boost the NAND Flash chip’s memory capacity “for free?” Well, why not?

There are a few reasons why not. First, MLC and TLC NAND Flash memory is slower than SLC NAND Flash memory. You need more time for the on-chip A/D conversion circuitry to resolve the amount of charge stored in the selected cell. Second, there are more and more complex wear, reliability, and endurance issues with MLC and TLC NAND Flash memory than for SLC NAND Flash memory. One NAND Flash wearout mechanism involves permanently trapped charge and MLC and TLC NAND Flash memories are more susceptible to such failures because the exact amount of charge trapped by the tunneling process is far more critical when storing more bits per cell. A little permanently trapped charge can really mess things up.

Choosing between SLC, MLC, and TLC memory can be tricky. The choice involves several of your design criteria including the simple and obvious one (read/write latency requirements) and the more fuzzy ones (failure rate, reliability, and cycle endurance). In short, you cannot choose using a simple cost/bit analysis.

Finally, if you’d like a painless video intro to the world of SLC, MLC, and TLC or 3BPC NAND Flash memory concepts, here’s a 5-minute video from Micron Technology:

• Solid-state disk drives (SSDs). With SATA and SAS interfaces, SSDs can plug directly into most existing systems and will provide an immediate performance boost.
• Flash caches. Located very near the DRAM, Flash caches provide fast ways to back up DRAM data over wide memory buses with high bandwidth and low latency. AgigA Tech’s AGIGARAM and Denali’s FlashPoint controller are both aimed at this NAND subniche.
• Specialty storage devices based on non-disk interface standards. Disk interfaces including SATA and SAS were developed with built-in assumptions about the drives they support. Those assumptions include some temporal assumptions based on having rotating mechanical memory. Those assumptions don’t apply to NAND-based storage devices so it’s possible to use interfaces with more bandwidth, PCIe and Hypertransport for example, to connect such storage and get better performance. This is the sort of product available from Fusion-io.

Which finally brings us to the trigger for this blog entry. The MIT/Stanford Venture Lab (VLAB) held a panel discussion at Stanford University on Tuesday, November 17 and the evening’s topic was “SSDs: Game-Changing Technology for Better, Bigger, Faster Applications and Application Development” and the first speaker was David Flynn, President and CTO of Fusion-io. Flynn’s talk contained many interesting and worthwhile things for followers of NAND-related topics as they relate to computer system design.

Early in his presentation, Flynn projected a photo of Charlie Chaplin playing one of the last great roles of his life, “The Great Dictator.” However, Chaplin’s roundish face had been replaced with an inset photo of a hard-disk platter and the caption was: “Getting rid of nasty Disc-tators.” Flynn emphasized that Fusion-io’s PCIe-connected products are not solid-state disks and they deliver more performance than solid-state disks because they are connected to a data pipeline that delivers more performance than existing disk interfaces. They are I/O-memory devices that provide 10x the capacity of DRAM per dollar, 50x the capacity of DRAM per “module,” and 100x the capacity of DRAM per Watt. Using these metrics, Flynn is making it clear that he understands the figures of merit valued by his company’s prospects.

Flynn then compared NAND Flash memory to aircraft aluminum. When metallurgists developed aluminum alloys suitable for aircraft, the entire airframe had to be re-engineered because aeronautical engineers couldn’t use aluminum as a direct replacement for wooden struts and dope-covered fabric. Aluminum ushered in new types of airframes that rapidly evolved. Aircraft performance soared as a result.

The same is true of computer systems (and software) developed before and after the Flash Zone is filled with something, whether it’s SSDs, Flash caches, or I/O-attached storage. Assumptions must be rethought and systems and software need to be redesigned to fully exploit the advantages of a populated Flash Zone.

There’s a similar but even larger performance gap between the access time of DRAM and HDDs. Although vendors have improved HDD capacity by 60% per year—each and every year—and HDD’s price per storage bit directly tracks that trend as well, there’s been very little improvement in HDD data transfer rate and interface speed and there’s been no dramatic change in HDD access time, which is largely determined by mechanical factors. Consequently, there’s been only a relatively slow improvement in HDD IOPS (I/O operations per second), which leads to a massive five-orders-of-magnitude (10^5) performance gap between DRAM access times and HDD access times and that performance gap is growing.

At the same time, DRAM’s volatility plays a role in a system’s sensitivity to power glitches and losses. When data is critical, and most data is critical these days, non-volatile memory just isn’t sufficient. Some means of retaining data through a power loss is usually required. In the past, HDDs have sufficed for non-volatile storage but they’re simply too slow these days.

The Flash Zone

Flash memory is a good candidate for filling this memory gap because it provides nonvolatile storage and it has become the cost-per-bit leader in semiconductor memory. Consequently, Mark Gogolewski, Denali Software’s CTO, calls this performance gap in the memory hierarchy the “Flash Zone” (see figure below and the Reference), the performance zone between DRAM and HDD access times.

The Flash Zone in memory hierarchy

The reasons for Flash memory’s candidacy to fill this gap include:

• In 2004, the per-bit cost of NAND Flash dropped below the previous category leader, DRAM.
• More NAND Flash bits shipped in 2005 than bits of any other type of semiconductor memory.
• More NAND Flash bits shipped in 2007 alone than all of the DRAM bits shipped in the last 25 years of commercial DRAM production.

There’s been a huge decrease in the per-bit cost of NAND Flash and a big capacity increase on a NAND Flash die. Consequently, NAND Flash memory fits nicely in the gap between DRAM and HDD. It offers faster access speeds than HDDs by at least two orders of magnitude while replicating an HDD’s non-volatile storage abilities. In addition, NAND Flash memory can draw considerably less power than HDDs when managed correctly. The opportunity for innovation in memory hierarchy is therefore huge.

Three Ways to Use NAND Flash: SSDs, Flash Cache, DRAM Backup

There are three ways to fill the Flash zone. The first approach is to use NAND Flash memory to create an HDD emulator using the same disk interface and possibly even the same form factor. Such drives are called solid-state drives (SSDs) and they have been gaining traction in the industry. Because they do not employ rotating memory, SSDs can deliver far faster access times than HDDs. However, there are costs associated with this approach. SSDs cost substantially more per stored bit than HDDs while retaining the overhead associated with HDD interfaces and protocols. The memory bus protocols and interfaces used to connect DRAMs to processors are much, much faster.

At this time, most analysts agree that NAND Flash memory will not overtake HDDs in cost per bit. Jim Handy of Objective Design presented the chart shown below at MemCon 2008 showing that the 25x cost-per-bit advantage for HDDs relative to NAND Flash memory cost per bit would continue for the foreseeable future.

NAND Flash and HDD cost-per-bit forecasts
(Jim Handy, Objective Design)

Denali’s memory market analyst Lane Mason recently commented that the pace of cost-per-bit reductions for NAND Flash memory will actually slow compared to price drops in recent years. So it doesn’t appear that NAND Flash will supplant HDD storage in the near- or medium-term future.

The second way to use NAND Flash memory in the Flash Zone is called a Flash cache. A Flash cache speeds access to an HDD by buffering the data stream between a processor and the HDD. Data is drawn from and written to HDDs as needed and the same data is simultaneously cached in NAND Flash. The next time this data is needed, it’s drawn directly from the Flash cache instead of the slower HDD. Flash caches do not require as much NAND Flash memory as SSDs, and therefore cost less, but they can deliver performance improvements when paired with HDDs.

The third way to use NAND Flash memory is to implement a backup strategy that allows the DRAM to operate normally when system power is available and to quickly save that data in non-volatile NAND Flash when system power fails. In this approach, which is used in AgigA Tech’s AGIGARAM Non Volatile System (NVS) modules, a backup power source provides the energy needed to safely tuck data away in non-volatile storage (NAND Flash), which then retains the data for a decade or more if needed.

This third approach to filling the Flash Zone offers several benefits including:

1. Fast backup when power fails
2. No energy required to save the data during power failure
3. Automated backup and restoration of data with no host-based software assist required

Which of these three approaches to use depends on the application (as always). If you’d like help deciding, please feel free to contact AgigA Tech.

Reference

The World is Flash: A Disruption of the Memory & Storage Hierarchy, Keynote Speech, Denali Memcon 09, Mark Gogolewski, CTO, Denali Software, Inc., www.denali.com