Techno Musings

The Storage Playoff on the Industry’s Proving Grounds

The pressure is on for team Performance Disks which lists on its roster 10Krpm and 15Krpm magnetic disk drives in various capacities with FC/SAS interfaces. The challengers are the  dynamic duo of FC/SAS interfaced Solid State Disks and high capacity – good enough performance – SATA interfaced drives. The playoff season in the storage industry though will last around 3-5 years before this prediction can be deemed true or not. The brackets look like this, high performance FC/SAS interfaced disk drives versus FC/SAS interfaced SSD, and a three way mini-playoff with SATA-based high capacity disks versus tape versus optical discs. The featured event of the storage playoff season will be between the high performance disks and SSD.

(My apologies, but using a football analogy is part of my therapy. My Chargers’ early exit from the NFL playoffs this week has left me in a sulking mood.)

Optical disc is the wildcard in the mix because every big capacity leaps seems to fall short from a content and archiving storage capacity requirement. This will yield yet another first round draft pick the next time around. One possible top draft pick may be the holographic center from HVD College (Holographic Versatile Disc). However, this may fall short on capacity again and be late to the game. While today’s specifications are impressive, the projected entry into the market for actual product delivery based on this technology may yield another disappointing career in the capacity starved storage industry league.

So my argument on the squeezing of 10Krpm and 15Krpm performance drives by SSD comes from the performance side of the problem. (At this point, I am referring to the standard 3.5 inch disk form factor.) While solid state storage isn’t a brand new concept, the more recent form factor, capacity, environments and cost has catapulted these devices into the mainstream and high performance computing datacenters. In fact, many HPC datacenters are in full throttle experimentation using these devices. Many (myself included) consider the HPC environments as the proving grounds or test track for pushing the envelope and previewing what will be commercial practice in the future. The primary points leading the way here are,

1. Performance – the obvious benefit is in read performance and greatly reduced  latency. We are in the rare instance where the interface and protocol speeds are behind the storage media’s performance.

2. Capacity – is not the issue. Most HPC architectures use the highest performing storage  (or lowest cost) as temporary working space. However, in order to deliver the highest throughput, many thousands of disks drives are aggregated together in the most demanding environments. Since you can’t buy 2GB magnetic disk drives anymore, there is a huge excess of capacity that typically goes unused. Basically, these disks are short-stroked to achieve the most performance with the remaining capacity either unused or used when the application isn’t  running.

3. Space – in order to achieve the same level of performance from SSD, far fewer devices are required which will require less space on the datacenter’s floor.

4. Power – even if the power consumption are equal between SSD and spinning  disks, fewer devices for the equivalent performance requirement will require less power to run and cool them.

5. Workload – the nature of most HPC applications is to stage data from a persistent  longer term source like a large archive or content depot, to the high performance storage pool directly attached to a HPC cluster. Typically, the application will read this data and crunch  numbers. Results could be written to the persistent storage. This workload is basically, write once – read many in this regard which is ideal for SSD’s duty cycle characteristics. This procedure is typically repeated over and over by removing the data and staging the next data set.

6. Cost – may not be as big an issue when looked at from a total cost of ownership perspective. Less capacity to manage and license for capacity that’s not used, less gear to manage and so on. I’m going to get with my colleague David Merrill and do some “storage economics” studies in this area.

So, high performance storage using SSD has the advantage, the persistent storage protecting the precious  source data in a content depot or active archive using high capacity low performance SATA interfaced magnetic disks, and a magnetic tape backend with hierarchical storage management moving data from tier to tier similar to my motto – “data in the right place, at the right time, for the right reason”. The statement of “slightly less performance” using high performance 15Krpm disk drives seems like a forced fit. While many HPC systems also employ high capacity SATA interfaced disk drives, the formula still holds, you need to aggregate thousands of these devices to reach the required performance level, but now you will end up even more unused capacity.

My friend and industry colleague, Arun Jagatheesan, and his team from the San Diego Supercomputer Center and the University of San Diego has been awarded the  Storage Challenge at SC09 in November 2009 for working on this exact approach.  Solving high performance computing and scientific computing problems in data intensive applications with SSD is just one area where intense experimentation is taking place. You can read more about this work and award at www.universityofcalifornia.edu/news/article/22439 .