Combining HDD and Flash in Computers Cures Many Issues

SDDs have tried to displace HDDs in computers for a few years but the higher cost of flash memory has been a major barrier to wide-spread adoption.  Lower flash memory prices will help adoption but HDDs decrease in $/GB at about the same rate as SSDs so the relative ratio of prices doesn’t improve much in SSDs’ favor.  At the same time there are serious issues in performance for many computers with HDDs, associated with the slower access time of HDDs. 

There have attempts to combine the advantages of HDDs and flasy memory in the past such as Intel’s Turbo Memory and the hybrid hard disk alliance but these were mostly  dependent upon the operating system to provide performance advantages.  The latest initiatives to combine flash memory and hard disk drives to create tiered storage systems in computers are known as Blink Boot, hyperHDD and the solid state hybrid hard drive.

Most of these approaches (hyperHDD and solid state hybrid hard drive) don’t depend upon the operating system to manage the use of the flash memory and the HDDs.  In the case of the recent solid state hybrid hard drive from Seagate the 4 GB of  flash memory  on the PCB board of the hard drive is used to store the most recently accessed data that the computer is using.  This is done internally by the hard drive providing a boost in access speed for this content without any special requirements on the computer operating system.

By adding a little flash memory to a hard disk drive for frequently accessed data or even for OS and application booting while still keeping the HDD for inexpensive mass storage makes a lot of sense.  Computer storage tiering with flash memory and HDDs could finally help flash memory become mainstream in computers.

Kaminario – A New Name for Solid State Storage

An Israeli start-up named Kaminario is attacking Texas Memory Systems’ home turf with a DRAM SSD that offers speeds as fast as 1.5 million IOPS.  While TMS has built itself a comfortable niche business using custom hardware, Kaminario’s K2 SSD, announced on June 16, is made using standard off-the-shelf low-profile blade servers from Dell.  Only the software is proprietary.

DRAM SSDs are an interesting product that serves niches which flash SSDs are unlikely to penetrate.  Objective Analysis’ new report on Enterprise SSDs explores the price and speed dynamics that separate these two technologies.  See the Objective Analysis Reports page for more information.

Some of the K2’s internal servers are dedicated to handling I/O, and are called “io Directors.”  The bandwidth of the storage system scales linearly with the number of io Directors used – a pair of io Directors provides 300K IOPS, and ten io Directors will support 1.5M IOPS.  Below the io Directors are other servers called “Data Nodes” which manage the storage.  Capacity scales linearly with the addition of Data Nodes.  Today’s limit is 3.5TB, but this number will increase over time.

Redundancy is a key feature of the Kaminario K2: There are at least two of any device: io Directors, Data Nodes, and HDDs per Data Node, since the DRAM-based data is stored onto HDDs in the event of an unexpected power failure.  The system can communicate with the host through a range of interfaces, with FCOE offered at introduction.

Kaminario’s K2 boasts a significantly smaller footprint and price tag than HDD-based systems with competing IOPS levels.

To find out more about Kaminario visit Kaminario.com

Violin Memory wants to Replace your Storage Array

Violin Memory introduced their 3000 series memory appliance in mid-May.  This million-plus-IOPS device piles 10-20 terabytes of NAND flash storage into a single 3U cabinet at a price that Violin’s management claims is equivalent to that of high-end storage arrays.

The system, introduced at $20/GB, or $200,000, is intended to provide enough storage at a low enough price to eliminate any need to manage hot data into and out of a limited number of small solid state drives.  Instead, Violin argues, the appliance’s capacity is big enough and cheap enough that an entire database can be economically stored within it, giving lightning-fast access to the entire database at once.

Note that Violin acquired Gear6 a month later, in mid-June.  This seems to reveal that the company is hedging its bets, taking advantage of a distressed caching company’s expertise to assure a strong position in architectures based upon a smaller memory appliance managed by caching software.

There is a good bit of detail about how and why both of these approaches make sense in Objective Analysis’ newest Enterprise SSD report.  See the Objective Analysis Reports page for more information.

But in regard to the Series 3000, CIOs whose databases are even larger than 10TB will be comforted to hear that Violin will be introducing appliances with as much as 60TB of storage by year-end.

Violin’s 3000 series can be configured through a communications module to support nearly any interface: Fibre Channel, 10Gb Ethernet, FCOE, PCIe, with Violin offering to support “Even InfiniBand, if asked.”  Inside are 84 modules, each built of a combination of DRAM and both SLC and MLC NAND flash, configured to assure data and pathway redundancy.

This high level of redundancy and fault management is one of Violin’s hallmarks.

Violin’s website is Violin-Memory.com

Nimbus: No Fast HDDs

San Francisco’s Nimbus Data Systems launched a solid state storage system in late April that is intended to replace all the HDDs used in a system except for slow disks used in near line storage.  Nimbus holds a viewpoint that solid-state drives eliminate the need for fast disk storage, and that in future times all data centers will be built using only SSDs for speed and capacity drives (slow HDDs) for mass storage.  This viewpoint is gaining a growing following.

Nimbus’ S-Class Enterprise Flash Storage System uses a proprietary 6GB SAS flash module, rather than off-the-shelf SSDs, to keep the costs low in their systems.  Storage capacity is 2.5-5.0TB per 2U enclosure, and can be scaled up to 100TB.  Throughput is claimed to be 500K IOPS through 10Gb Ethernet connections.   Prices are roughly $8/GB.

Although Nimbus previously sold systems based on a mix of SSDs and HDDs, they have moved away from using HDDs, and expect for data center managers to adopt this new approach.

There’s merit to this argument, but it will probably take a few years before CIOs agree on the role of NAND flash vs. enterprise HDDs vs. capacity HDDs in the data center. There’s a lot more detail on the approaches being considered for flash in the enterprise data center in Objective Analysis’ new Enterprise SSD report.  See the Objective Analysis Reports page for more information.

You can find out more at  NimbusData.com

New Article: Solid State Drives for Energy Savings

A new article, co-authored by myself and Tom Coughlin, can now be read from the SNIA Europe website.  “Solid State Drives for Energy Savings” explains the energy benefits that are being discovered when IT managers start to bring SSDs into their data centers. 

The article is a quick two pager, and it introduces SNIA’s new TCO Calculator (Total Cost of Ownership), a clever tool that helps estimate the power, rack space, and other savings that come along with a conversion of fast storage from enterprise HDDs to SSDs.

[Update: After clicking on the above link, it will be necessary to download the April 2010 edition of  Storage Networking Times, in order to read the article.]

Understanding Flash Impact on Enterprise Storage Architectures

In London this week, I was invited to present on solid state storage at the SNIA Europe Data Center Academy event.  It was  a well-attended event with a good mix of  IT managers, consultants and press in the audience – including Chris Mellor from The Register who subsequently published this blog.  Chris is one of the most perceptive journalists in the industry and really nailed the key issues in his blog.  Check it out!

BTW, the presentation is among the latest batch of SNIA tutorials, which you can find here.

Capacities and Storage Devices

Some SSD advocates project that SSD price per gigabyte will cross over that of HDDs, due to slower growth in areal density of HDDs in the future than it has grown in the past.  HDD price per GB declines will slow as a result of a slower areal density growth.  The argument is that this would allow flash gigabyte prices to blow past HDD prices just as they slipped below DRAM gigabyte prices in 2004.

 Some of these advocates recently predict that 3.5″ HDDs will “only” reach 6TB by 2015.  Although we find it likely that we will have 6 TB HDDs in mass production by then, Coughlin Associates expects to see 10TB maximum announced product capacity of 3.5-inch hard disk drives by that time. 

 Slower areal density growth of hard disk drives may result from transition difficulties to new recording technologies such as patterned media and heat assisted magnetic recording.  It appears likely that we may see areal density growth slowing from 40-50% annually today to 20% or possibly even less over the next few years.  Today the HDD industry is shipping 2 TB 3.5-inch HDDs and 1 TB 2.5-inch HDDs and will likely ship 3 TB or larger (3.5-inch) drives in the second half of 2010.  If the areal density of HDDs increased only 20% annually from 2010 through 2015 this would give us 7.5 TB 3.5-inch HDDs and over 3 TB 2.5-inch HDDs.

 Although a slow down is likely during a technology transition phase it is likely going to slow down gradually from today’s roughly 40% annual areal density growth rate.  So let’s say we have one more year of 40% growth (2010-2011), one year of 30% growth (2011-2012) and then 20% growth for the three remaining years to 2010.  With a 3 TB capacity in 2010 that would give us a 9.4TB capacity in 2010.  There is enough uncertainty in these numbers that the actual capacity could be between 8 and 11 TB so let’s say the maximum storage capacity in 2010 for 3.5-inch drives is 10 TB.  Likewise because of the geometry differences the maximum 2.5-inch storage capacity would be about 5 TB. 

 If the HDD industry stays true to its history, these 10TB HDDs will cost $50, giving a price per terabyte of $5.  Meanwhile, NAND flash terabyte prices will have declined to $50-100, preventing SSDs from displacing HDDs at least through 2015!

Solid State Storage Initiative – Transition in Leadership: Looking Back and Going Forward

SSSI and SSS TWG:

The SNIA Solid State Storage Initiative (SSSI) recently noted the transition in leadership from founding Chair Phil Mills of IBM.  Phil Mills, Secretary of the SNIA Board of Directors, drove formation of the SSSI in late 2008 and was instrumental in developing the Initiative’s organizational mission and objectives as well as forming the associated SNIA Solid State Storage Technical Working Group.  Due in large part to Phil’s Herculean efforts, the SSSI was successful in attaining critical mass by recruiting 26 founding members – a number that continued to grow to the current membership of 34 SSSI member companies, along with 56 SSS TWG member companies.

SSSI Mission:

The founding mission statement of the SSSI articulated a dedication to “foster the growth and success of the solid state storage market for client and enterprise applications… that encompasses marketing outreach, education, collaboration with SSS industry standards bodies, development of SNIA SSS Specifications and Standards (such as the SSS TWG), and a close following of advancements in non-volatile memory for solid state mass storage.”

Progress to Date:

Taking stock of the SSSI at this transition point, it is clear that the SSSI has been very successful in effecting these goals.  The SSSI has active committees for Marketing, Business Development, Education, Technical Development and SSS Total Cost of Ownership.  In conjunction with the work in the SSS TWG, the SSSI has made great strides in all areas.  Deliverables achieved to date include establishment of the SSSI as THE authoritative voice on Solid State Storage Performance, collaboration and cooperation with other Standards groups and Trade Associations (JEDEC, SSDA, and others), completion of several key white papers and tutorials on many aspects of solid state storage, industry noted presentations and presence at trade shows and events, completion of a TCO calculator for SSS, on-going investigations in emerging areas of SSS technologies (such as “drive pairing” and “storage tiering”) and imminent release of the SSS TWG Performance Test Specification for Public Technical Review – the industry’s first SSS Performance Specification whose goal is to help standardize the nomenclature, metrics, methodologies, tests and reporting of SSS performance.

Looking Forward:

Phil leaves the Initiative at at time of great market activity and leaves in place a capable team to carry the SSSI forward.  The SSSI Governing Board has appointed Paul Wassenberg of Marvell as the acting SSSI Chair.  Paul has been the chair of the TechDev sub committee and allows for a seamless transition with Paul’s extensive interaction with Education, Busdev, and the SSS TWG.  Paul and Marvell are also one of the founding companies of the SSSI and has been instrumental in getting the Initiative to where it is and will be key in moving us forward.

Come Join Us:

The SSSI continues to evangelize Solid State Storage and will have numerous opportunities to contribute to standardization of the SSS industry and widespread adoption and deployment of SSS mass storage.

Thank You Phil:

The entire SSSI wants to thank Phil for his dedication, leadership and vision and hopes to take the SSSI to the next level.  In that vein, the Initiative actively seeks to expand its membership and to address topics of interest to the SSS community and the Initiative’s membership.  People and companies that have an interest in participating in this exciting industry are invited to visit the SSSI at http://www.snia.org/forums/sssi

SSDs Strong at MySQL Conference

The MySQL Conference, a gathering of programmers who share database strategies, was held in the Santa Clara Convention Center this week.  One hot topic was SSDs.

My favorite session was hosted by Fusion-io.  They rounded up four satisfied customers who discussed how Fusion-io SSDs had benefited them.

Craigslist, found that in their systems the RAID card was their highest point of failure.  That doesn’t help much, when the whole point of a RAID system is to prevent failures!  Although the company is only halfway through their first SSD deployment, they are recognizing 2/3 power reductions while their tests are running surprisingly faster than they had with RAID systems.  SSDs are allowing them to move to two incidences of MySQL on a single server with two IOdrives on each server.  Their bottleneck has moved from storage to the software.

Cloudmark, a net security firm that blocks Spam for over 1 billion in-boxes, started with a system that consumed 48U of servers and storage, and their needs ballooned to the point that they were worried they couldn’t keep pace.  By adding SSDs they were able to actually reduce their systems to 17U, losing 180 drives along the way.  Their processor CPUs are now operating at maximum capacity, something they have never seen before.  Cloudmark’s present system runs 22Us of systems with 22 IOdrives operating in a RAID configuration that is significantly cheaper than a SAN.  The SAN costs $500K.  They have been able to replace this with two servers, each with an IOdrive, for about $20K all told.

Answers.com: The 18th-rated Internet site in the US and 31st worldwide, is a staunch user of HP hardware.  When HP announced their support of the IOdrive, Answers.com bought a couple and found that they could avoid the purchase of four additional servers by buying a 320GB Fusion-io drive for the price of about two servers.  When they compared performance against their SAS-based systems Answers.com was astounded to get a ten times improvement in complex queries from 350 to 3,500 per second.  Restoring the system from backup dropped from over 6 hours to 12½ minutes!  CPU loading dropped from 30% to 18%.  The company’s old topology had five data centers with four servers per data center.  Today a single server per data center does the trick.  Their tests indicate that they could reduce the number of servers to 1/9th the original number, but conservative policies prevent them from trying this.  The current configuration has been in place for one year, and the company still has more processing power than they need.

Percona is an important MySQL consulting firm, helping clients through system analysis, coding, and even training.  Percona ran extensive tests on Fusion-io and Intel mainstream SSDs in comparison with enterprise HDDs and even HDD RAID systems.  The conclusion was that the Fusion-io IOdrives were a hands-down winner in database applications.

All in all these four users present an extremely compelling case for SSDs in general, and Fusion-io drives in particular (although we should keep in mind that Fusion-io selected the speakers for this panel.)  With the results these firms experienced it’s clear that now is the time for all data center managers to stand up and pay attention!

Spring SNW – Recap

Last week’s SNW show in Orlando had encouraging signs that the recession is receding. IT end-user attendance was slightly up on previous shows (although vendor attendance was down); IT end users had budget and were looking for the optimal way to invest it, and there was clear excitement around a number of topics – including solid state storage.

Solid State Storage was well represented on the agenda, both in the vendor-neutral SNIA Tutorials, and in the SNW sessions:

SNIA Tutorials:

Executive Overview and Current Topics in Solid State Storage, Rob Peglar, Xiotech
Apples to Apples, Pears to Pears in SSS Performance Benchmarking, Esther Spanjer, Smart Modular Technologies
Latency: the Heartbeat of a Solid State Disk, Levi Norman, Texas Memory Systems
The Benefits of Solid State in Enterprise Storage Systems, David Dale, NetApp
Using SSD MLC NAND in Data Center Applications, Tony Roug, Intel
Solid State Storage Architectures, Jamon Bowen, Texas Memory Systems

SNW Vendor Presentations:
Panel: Is SSD Right for your Enterprise? Moderator: Jeff Janukowicz, IDC; Panelists: David Dale, NetApp; Steve Johnson, LSI Corporation; Gary Tressler, IBM; Rich Vignes, Seagate; Tom Lattin, HP
Dynamic Tiering – Taking Advantage of the Best of HDDs and Solid State, Ron P. Bianchini, Avere Systems
Getting the Most out of SSDs- Data Center IT System Optimization Best Practices, Mike Chenery, Pliant Technology
Flash Solid State Storage reliability & data Integrity – Are We There Yet? David Flynn, Fusion-io

All of the presentations are available on the SNW website. In addition, the SNIA Tutorials are available on the SNIA website.

Check them out.