Techno Musings

Talking About Storage Virtualization and Storage Federation and Storage…

So, while reading Devang Panshigar’s recent blog post (here) on his assessment of EMC’s Storage Federation concept, I like that he is contrasting HDS’ approach with Storage Virtualization. However, I do take exception to the statement of “While HDS has a slightly different approach yielding almost the same result using Storage Virtualization, HDS approaches storage federation in its USPV platform…” Since the USP-V is the second generation of our Storage Virtualization technology, it would seem to me that the statement should more appropriately read as “While EMC has a slightly different approach than HDS yielding somewhat similar results…”. I’m just being picky with the wording and chronological order of the statement here, false starts notwithstanding.

I do believe that virtualization is a great way to reduce storage management complexity overall in the datacenter and beyond. I also still like my infrastructure viewed as a stack of layers. The features of the lowest layers bubble up to the upper layers, for the most part, but are invisible in its behavior to the upper layers. For example, Dynamic Provisioning with the USP-V, even on any virtualized storage, benefits the upper layers of the stack in capacity provisioning and performance layout with no intervention. In fact, many of our customers are standardizing on HDP for every aspect of their storage needs because the automatic performance layout is as good or better than can be configured manually, and with far less “tuning” effort.

So I’m talking quite a bit about Storage Virtualization as oppose to Storage Federation. I can say that there’s not that much difference. It’s more like comparing Fuji Apples (Hitachi) to Granny Apples. Both are apples. If the support of distance as part of the distinction that separates storage virtualization from storage federation, then please read my last blog posting on Horizontal-UVM. In that use case, I discuss a copy-less storage distribution model using UVM in a different way. The best part is that UVM is has been in production for over 5 years. If it sounds like I’m implying that more innovation is coming, I’m not saying…

Devang poses a very good and interesting list of questions. I’d like to comment on some of them with my thoughts on this subject. To keep this more in line with reality, I’m going to limit my responses to within a 5 year future window.

Is storage federation that important?

Yes, for a given set of use cases. Maybe, for other use cases, while still other use cases could see a negative impact. Assuming technology continues to advance, the rate and timing are always inaccurate, storage virtualization and federation could see wider adoption as a way of increasing efficiency and lowering cost. Virtualization is already in most of our customer’s datacenters solving all of their problems.

Is storage federation the future of all storage?

Again, it will depend on the use cases. Also, the state-of-the-art tends to dictate what is possible and useable as well as when.

How large, a single namespace would you like to have? I believe HP IBRIX brings a similar concept of scaling storage to 16PB’s total in a single name space.

I do have questions about this question. To me this is virtualization at the file system layer and while I’m a big fan of Ibrix’ technology, I think these are two separate discussions. However, it does bring up the notion of very large single namespaces, be it block or file based. The analogy between the two could be viewed as LUN and file system. How large should a single LUN be and how large should a single file system be? Again, based on the state-of-the-art, these answers continuously shift forward usually based on performance and cost. No point in have a single 10 peta-byte LUN if you can’t format it in a reasonable amount of time, etc. However, given enough advances in speed and cost improvements, it will be possible someday. By the way, with a single 8Gbps Fibre Channel streaming at full bandwidth, it would take 145 days to format a single 10 peta-byte LUN. Your results may vary.

Will this technology shape the future of how we do computing today by leveraging and pooling storage assets together for a higher availability and efficiency?

Here at HDS, we already know the answer, yes. Except that we launched this technology over 5 years ago, so basically the future is now. We have been doing storage virtualization for over 5 years, and pooling and provisioning for almost 3 years. This means that the USP line has been unifying storage management, abstracting underlying storage, transparently tiering, adding features to legacy and featureless storage arrays, and leveraging our customer’s investments far beyond their anticipated expectations, for a long time. This also means we know what are customers deploy and how they want it to solve their problems. Revealing what we are coming up with next is beyond the scope of this posting, but stay tune.

Last question for now.

Is EMC Virtual Storage just catching up to HDS technology? Or is VPLEX vision a big and unique one that will change the direction of EMC Storage in the future…

This one is sort of unfair and loaded. The answer is no, EMC has not caught up to HDS, though they will continue to try. Problem for EMC is that HDS has such a long head-start and an in-depth knowledge of this space, that it may be impossible for them to catch-up. I say “may be impossible” because I can think of one way for them to instantly achieve par with HDS, kill their virtualization strategy (again) and resell the USP-V.