It helps to Clarify
by Hu Yoshida on Feb 25, 2007
Tony Pearson posted “Hu Yoshida should know better” in response to my blog on storage virtualization and vendor lock in. This blog caused him to dirty his computer screen. Sorry Tony, I think I need to give you an explanation. I admit that the sentence that you quoted in your blog was a little alarming, but I didn’t intend for you to lose your coffee over it.
I agree that storage virtualization in the all encompassing sense that you defined, started many years ago. In fact the first RAMAC disk back in 1956 used a servo control to locate records. This could be considered an early form of virtualization. There is no doubt that IBM has contributed greatly to virtualization of storage with RAID, and Virtual tape servers. Other vendors like Veritas with volume managers, STK with log structured files, and EMC, who contributed global cache controllers with their ICDA (Integrated Cache Disk Array) back in 1990, all contributed to storage virtualization.
The first requirement for any storage system is protection of data and high availability. Enterprise storage systems like the IBM DS8300, the EMC DMX, and the HDS USP/NSC are designed with this in mind. They have multiple processors that access a large global cache, so that even if one or two processors fail, the system will continue to perform with no data loss. These control units also have knowledge of track tables and cache slots, and can take snapshots of volumes, or replicate consistency groups of volumes across synchronous and asynchronous distances. These control units create LUNs out of RAID array groups. These are all forms of virtualization.
The enterprise control unit has been doing storage virtualization from the very beginning. Up to now it has been doing it to its own proprietary disks. The USP simply extends it to any disk that has a FC standard interface. There is no need to introduce a whole new layer of management complexity into the SAN in order to do storage virtualization. There are some things you need to add to the storage control unit like virtual ports to enable scaling to thousand of host connections, host storage domains for safe multi-tenancy, virtual partitioning to ensure quality of service, a switch architecture for scalability, and separation of control and data for dynamic configurability, performance, and security; but the basic mechanisms for storage virtualization are already being done in the storage control unit.
The SAN Volume Controller is a Volume Controller. It is not a Storage controller. The IBM DS8300 is a Storage controller and there is a world of difference between that and a Volume Controller. Although the SVC has multiple controllers and cache memory, each controller has a separate memory, and not a global shared memory cache as the DS8300. Each SVC controller is also limited to 4 GB of cache and 4 FC ports. As a result they can not provide the availability, scalability, and performance of a DS8300. If they could, there would be no need for a DS8300.
When the concept of storage virtualization was re-introduced around year 2000, it was introduced for the purpose of volume management. Some of the early literature focused their justification on the elimination of license costs for host based Logical Volume Managers. Today, the value of storage virtualization resides in the ability to provide common storage services, like distance replication, and non disruptive movement of data across tiers of storage. These types of storage services can only be provided by an enterprise class storage control unit.
That is what led me to say that “storage virtualization can only be done in a storage controller”. I should have said that the virtualization of storage services can only be done in a storage controller. I hope this helps to clarify my earlier statement