What defines an enterprise storage system?
by Hu Yoshida on Sep 11, 2012
Enterprise storage systems are multi-processor storage systems with a global cache memory. The global cache memory enables a LUN to be mapped to host servers and accessed through multiple port processors that are assigned to that host server. By supporting multiple active paths between the host server and the LUN image in cache, throughput and availability can be improved and quality of service can be maintained even if one of the paths is down.
After my series Redefining Unified Storage, I was asked if Hitachi Unified Storage (HUS) could be considered an entry enterprise storage system due to its ability to support load balancing across its two controllers. My response is “no.” Unlike Hitachi VSP, HUS does not have a global cache, which means that each controller has its own cache. The LUN is processed through one cache or the other, and HUS manages the paths so that access is assigned to the controller, which has the cache image of the LUN. The load balancing occurs automatically when the difference in utilization between the two controllers exceeds a certain threshold and then the LUN image is transferred over to the less active controller. This greatly simplifies the operation and management of a dual controller system since it eliminates the manual task of assigning LUN ownership and rebalancing the LUN ownership when the activity against the controllers gets out of balance. However, this does not provide the availability and performance of an enterprise global cache storage system.
Another characteristic of an enterprise storage system like VSP is its high availability and performance, which comes from its hybrid architecture that enables multiple CPUs and ASICS to access the same LUN image in cache. This enables functions to be added to the storage system without impacting I/O performance by distributing the workload across CPU cores and ASICs. In traditional dual-controller storage systems the controllers have to manage all these functions, and performance declines as more and more functions like replication and tiering are added to the storage system. If one of the controllers goes down than half the resources in a dual-controller system are lost and the remaining controller becomes a single point of failure as it has to take on the workload of the other controller. HUS is more like an enterprise storage controller in that it introduces a distinct separation of controller functionality into front-end and back-end I/O engines. HDS proprietary DCTL processors are at the core with Intel I/O processors on the front end, with Tachyon FC chips and separate I/O processors on the back-end for SAS, SSD and SATA-II disks. This design is responsible for the high performance of HUS. However, when a DCTL goes down, all the workload shifts to the remaining DCTL, which becomes a single point of failure. In VSP the CPUs, front- and back-end ASICs, and cache modules are installed in pairs and if one of the CPUs or ASICS go down VSP can continue to run and rebalance workload across the remaining CPU cores, ASICs, and cache through an internal switch matrix architecture. Multiple pairs of CPUs, ASICs, and cache modules can be added to ensure no single point of failure.
The internal switch matrix is unique to HDS. Other enterprise systems may have a global cache, but the cache and control processors are contained in a controller node and the nodes are connected over an external switch or backplane. If an I/O request goes in on one node and the data resides in another node, the data has to be routed across the external connection to the other node for processing and then passed back to the originating node for delivery to the host. If a controller node goes down, all the resources in that node are lost including front- and back-end ports, cache and processors. While other nodes can take up the workload, you lose half your resources in the case of a two-node cluster.
There are storage systems on the market today that claim to be entry enterprise storage systems. They are applying this term to loosely coupled modular storage systems. These are not tightly coupled multi-processor enterprise systems with a global cache. These are still dual controller modular systems where each processor must do all the work of managing the front-end ports to the servers, the back-end disk adapters, the local cache, and the additional coupling interface to the other modular processors. While a server’s workload can be passed over the coupling interface to another modular processor, the net is that only one processor can process a server’s workload. In an enterprise storage system, multiple processors will work together to process a server’s workload because all the processors work with a single global cache image of the LUN.
Why is enterprise storage important? The main driver is the scale up consolidation that is occurring in the server virtualization space. Servers are going to multi-core processors with simultaneous multi threading, operating systems are being virtualized and consolidated with VMware and hypervisors, and server bandwidths are increasing to 8 Gbps with FC and 10 Gbps with Ethernet. Storage systems must be able to scale up to meet these requirements. In addition more and more workload is being pushed down into the storage systems. Functions like moves, copies, replication, wide striping, thin provisioning and heterogeneous virtualization is driving more workload to the storage systems. Only tightly coupled storage systems that can scale up as well as scale out can support these types of requirements.
So there really is no storage that can be called an entry enterprise storage system today. It is either enterprise with scale up and scale out capability or it is loosely coupled modular storage systems that can only scale out. However, watch this space because there is a market need for entry enterprise storage systems with the functionality I described above and I expect we’ll see further developments in this area.
Comments (6 )
Interesting point of view, but with all respect that I have for you, I don’t agree on all points. 3Par Storage Systems can be considered High-End Enterprise Storage, don’t monolithic, but Hi-End yes.
The HP 3Par supports up to eight controllers (nodes) and have Persistent Cache and Full-Mesh Architecture. It also support truly active-active symmetric LUN Access.
I see 3Par P10000 really like a true Entreprise Storage, not a entry enterprise.
I would like to know your point about 3Par market positioning, if you can share this.
Hu, interesting points. I’m just a bit confused. In the first sentence you seem to present a fact (“Enterprise storage systems are multi-processor storage systems with a global cache memory.”) and the whole article is based on it. But where do you take this “fact” from? Is this out of an official definition for Enterprise storage I’m just not aware of? In my opinion the term Enterprise storage is not tightly coupled to a set of features (let alone just “the features my system has”). That’s a bit like someone would have said “A public transportation vehicle must consist of a perfectly handcrafted spring-borne carriage drawn by two to six purebred horses.” and then suddenly someone invents the car and there are cabs and busses. So in my eyes: If a storage system is used by enterprises for “enterprise applications” with all the required IOPS, data rates, stability and HA/DR, why not call it Enterprise storage?
(This is my own opinion and not necessarily IBM’s…)
There is a mistake on the picture.
HP 3PAR can have more then two controllers and is considered as enterprise storage.
Thanks for your comments. There really is no industry accepted definition for enterprise storage. This is my position. All data center class storage systems require IOPs. data rate, stability, and most cases HA/DR. What is different about enterprise storage is the ability is to sustain performance, availability, and reliability when components fail and need to be repaired non-disruptively. Enterprise storage must also be able to scale up as well as scale out. Modular storage systems suffer performance degradation and availability exposures when components fail. A global cache with multiprocessors would be able to sustain performance and availability. Modular storage systems also do not scale up. They can only scale out through clustering across multiple storage controllers. Again a global cache with multiple processors would enable the ability to scale up. In my view, enterprise storage systems that fit my definition are storage systems like the Hitachi USP, USP V, and VSP, HP XP 9500, EMC DMX and VMAX, and the IBM DS8000.
Thank you for calling this to my attention. Although 3 PAR does have more than two controllers, it is a cluster of two controller nodes and does not have a global cache. However, the HP XP9500 does have a global cache so I have updated the image to include the XP 9500.
Hello Diego, my reason for not classifying the 3 PAR storage as an enterprise offering is related to scalability and availability.
3 PAR is a two controller “midrange architecture” which does not support a global cache. Although 3 PAR can be configured with 2, 4, 6, or 8 of these two controller nodes, clustered together through an external mesh architecture, I/O is processed through one controller in one node. Although the aggregate configuration maximums can get to very large cache and large numbers of disk and FC connections, the processing is done by one controller. It does not scale up by combining the resources of all the processors, cache modules, disk and FC connections. From an availability standpoint I believe that the replacement of one disk requires taking all the drives in the 4 drive magazine offline and replacement of a cache card requires the entire node to be taken offline.
The 3 PAR vendor does have an excellent enterprise storage system in their portfolio, the XP 9500, which is described on their website as ” Delivers bulletproof storage for the mission-critical Converged Infrastructure (CI) where constant access to data is required—even in the event of a disaster”