A debate often arises between, server, network, and storage vendors about where “intelligence” should reside. “Intelligence” in this context is associated with virtualization. The simple answer is that intelligence needs to reside at all three levels. However, intelligence needs information to act upon in order to provide value or functionality like virtualization. The question should not be “where should intelligence reside?” but rather where does the information reside so that intelligence can be applied to provide a function like virtualization most efficiently.
Different levels in the I/O process have access to different types of information. The server has information about applications and server platform state, so that is where server virtualization needs to reside. A very successful product in this area is VMware. Filers have information about files and mount points, so that is where NAS virtualization should reside. A SAN switch has information about the state of the network, World Wide Names, and access to FC headers to know where packets go, so the switch is where SAN Virtualization needs to reside. Cisco’s VSAN is an example of SAN Virtualization. The Storage controller has information about the I/O command since it is the target for the application I/O. The Storage controller knows the status of storage port mode sets, which I/O commands to execute, which SCSI code pages are needed, and the location of RAID groups, disk track slots and cache tables. Therefore the storage controller is where storage virtualization can be executed most efficiently. While this seems logical this flies in the face of a whole emerging industry around storage virtualization in the SAN.
Since a SAN sits between the servers and storage and connects one to another, many people in the storage industry assumed that this is the ideal place for storage virtualization. Unfortunately the SAN only has access to the routing information in the FC packet header and the state of the SAN which is contained in the Simple Name Server in each switch. It has no information about the I/O itself which is contained in the FC packet. There are only two ways for the SAN to see this information: act like a proxy target to the host, or crack open the FC packet to see the Content Directory block. Both approaches violate the original intent of FC, to keep the contents of the FC packet secure and avoid spoofing of WWNs.
Early attempts at storage virtualization in the SAN were done with Intel appliances, sitting in the network, acting as target proxies to the application I/O request. These appliances used target mode HBAs to receive the I/O, defined new logical LUNs, remapped the address, and then reissued the I/O through an initiator HBA to the storage controller where the I/O commands were executed. The status of the I/O had to be sent back first to the appliance and then to the host which added latency to every I/O. Another approach was to put an agent into the host to intercept the I/O and send the target address to an IP attached appliance for remapping before it was issued by the host HBA. This added less latency, since after the remapping of the target was done initially, the rest of the I/Os to that target went directly through the SAN to the storage controller. The appliance that did the remapping of the target sat outside the SAN. This led to a whole new set of discussions and arguments around the advantages of “inband”, where the remapping was done by an appliance sitting in the FC path, and “out of band” where the remapping is done by an appliance sitting outside of the FC path. I can’t get excited about these distinctions. Actually they both sit between the application or host “initiator” and the storage “target” and perform unnatural acts in order to get at information which was never intend to be available to the SAN. The Job of the SAN is to provide a secure, efficient, in order delivery, of FC packets to a designated FC port defined by a unique World Wide Name.
Switch vendors sensing an opportunity to expand their market, jumped into the virtualization wars, buying up companies like Andiamo and Rhapsody who had technologies that enabled the switches to trap FC packets as they came through the switch, crack them open and send the contents to an application blade in the switch to remap the target. Essentially they were off loading the agents that used to do this in the host and consolidated that function on to special switch ports. This spawned a whole new standards effort to define a Fabric Application Interface standard so that the switch (fabric) vendors could support the same functions with different hardware and software implementations. For recoverability, all these appliances and switches required clustering. Clusters add expense and additional complexity. While it protects against complete failure when one node fails, the other node has no protection and a maintenance window needs to be scheduled to repair the cluster. Clusters also have separate memories which need to be kept consistent. If the content of the memories differ, which one has the right information? This adds complexity to an already complex SAN environment. How do you integrate this into a SAN environment where every switch must maintain the same state change information in their SNS tables, handle buffer credit decongestion, SAN zoning, VSAN headers, path latencies, ISL chatter, etc. All this effort with billions of dollars in investment and R&D, and additional layers of complexity was required to discover what target and LUN the host was trying to address to enable storage virtualization in the SAN. Investors, sensing the opportunity to make money off of IPO’s, added to the hype.
There is no need to create this whole new technology platform since storage controllers already do storage virtualization. All the major storage vendors provide virtualization of storage in highly available storage controllers with a global cache which is accessed by a large number of physical and logical FC ports. This is a processing configuration which provides an order of magnitude more availability and scalability than any cluster solution sitting in the SAN. They create LUNs (or logical devices in the case of mainframes) out of RAID groups which is another logical representation of physical devices. There is no need to trap FC packets, extract the target id and remap it, because they are the intended target to the host. All the information is available to the storage controller. Virtualization, the mapping of I/O to physical devices is done behind the controller, rather than in front. It is certainly less complicated. All that is required is to attach external storage behind the storage controller through a standard protocol like FC. The storage controller queries the FC attachment and discovers and visualizes the LUNS in the external storage and presents them to the host, just as it would with its own internal LUNs. No additional layers of complexity are required, no new standards, no discussion of “in band” or “out of band”, and no new SAN requirements. In fact there is no dependence on the SAN. This approach can present external storage to any host that can attach to the storage controller through DAS, SAN, NAS, FC, ESCON, and FICON.
In addition to storage virtualization, this approach essentially makes all the functionality that already exists in the controller available to the externally attached storage. SAN based solutions must reinvent all the data mobility functions that exists in most high end storage controllers and move them up onto the network appliance or switch blade. This becomes a major task for SAN based storage virtualization since the information they need to do data mobility functions, like remote replication, are not readily available in the SAN. That information such as disk track slots, block displacements and cache segments reside in the storage controller. They are also limited in the amount of cache they have for holding data while waiting for response of a successful completion. The distinction for storage controller virtualization is that no additional agents or proxies are needed since they have all the information that is needed to do the job. This is the approach which Hitachi has taken with their TagmaStore controllers, the USP and NSC.
If it is less complicated to do all this in the storage controller, than why don’t other storage vendors do the same? All that is required is the separation of the storage controller function from the disk array so that other vendor’s disk arrays can be attached. Perhaps it is the fear that this would cut into their revenues since it would commoditize their disk arrays. Hitachi considered this, but decided that sooner or later someone would disaggregate the storage controller from the disk array anyway. Some storage vendors have gone the other way, creating larger monolithic storage array controllers in order to sell more disks with each controller. I believe that is a fool’s errand. The disk capacity prices are declining 30% to 35% per year due to advances in magnetic recording technologies. It is a commodity, but storage vendors have been bundling them with storage controllers which forces IT to buy larger and larger disk arrays at up-front prices. By separating the disk array from the storage controller, IT can buy only the capacity he needs, when he needs it, from whoever has the best value, and can take advantage of the price erosion of the commodity disk.
Separation of the storage controller from the disk array does require an architecture that enables dynamic cache configuration changes that can scale to thousands of heterogeneous host connections, manage peta bytes of storage, and provide logical partitioning and host storage domains to guarantee safe multi tenancy and quality of service. That is where the effort needs to be placed to achieve storage virtualization.
Put the storage virtualization intelligence where the information exists, in the storage controller.
Comments (0)
