Tiered Storage with virtualization in place or in the cloud
by Hu Yoshida on Aug 23, 2009
There has been a lot of discussion around tiered storage after my last blog on that subject. Stephen Foskett and Carter George have posted, suggesting different approaches to tiered storage. All of us agree that unstructured data and content data are making up a larger portion of our storage requirements and we have to find ways to lower the cost of storing that data. Carter claims that most storage administrators know that 80% of their storage could be stored on lower cost tiers of storage.
Stephen suggests that cloud storage is a great alternative for lower tier storage considering the “massive cost” of onsite disk storage. I think “massive cost” is an over generalization. The total cost of onsite storage can be greatly reduced through the use of virtualization, tiering, dynamic provisioning, and de-duplication technologies. Having this type of storage onsite gives you the flexibility of making copies to lower cost tiers for other business uses, dynamically provision storage as your business requires, control and ensure protection from the bad behavior of other users which is the exposure you have in a shared services model like the cloud.
With most cloud storage services you agree up front to the capacity and level of service that you will need over a certain period. If you over estimate you are paying too much, and if you underestimate you usually pay a penalty for exceeding your SLA. The problem is that storage is stateful. It is not stateless like processing power, and bandwidth. Some one has to bear the inventory carrying cost and agreements must be made around projected usage to minimize that cost. How good are you at projecting usage? If you use a cloud service, do you pay for the full capacity without regard to thin provisioning and de-duplication savings?
However, I agree that this could be an alternative for some types of content data storage. There are archive, backup, and infrastructure cloud services that can store data offsite. One needs to run the numbers and consider the total cost of ownership around these services. While this may change capital expense into a pay as you grow model, there are additional expenses or risks around a shared services model in terms of performance, accessibility, security, compliance, and long term viability. While you can offload the capital costs, you can not offload your responsibility for records management, data availability, compliance, and security.
Carter suggests that data can be tiered without actually moving it to another physical tier. Carter is the co-founder of Ocarina networks. Ocarina ECOsystem can create multiple virtual tiers within a single filer or volume with multiple dedupe settings and multiple levels of compression without moving the file or volume off the tier that it started on. While this may have a performance impact, this may not matter for the type of data that would go to lower tiers of storage. It can also be used with physical tiering that is provided dynamically through the USP V. Virtual tiering in place on tier 1 storage, especially if it is expensive flash drive storage, may not be cost effective. However, combining physical tiering with virtual tiering in place on lower cost tiers of storage may be very attractive depending on the incremental cost of the software.
Tiering is definitely the way to reduce the cost of storage, especially today when there is a wide difference in price and performance between Flash disks and SATA disks. Before virtualization, the cost and disruption of moving data across storage tiers was prohibitive. Now there are many ways to do this with virtualization whether it is with storage, or volume, or file, or infrastructure as in cloud. The days when everything was stored on one expensive tier of storage or locked into multiple static tiers of storage are over.
Comments (2 )
Interesting post, Hu and thanks for responding to my thoughts on this topic. Just to give your readers a little bit more background, Ocarina works transparently with HDS’ file services solutions, HNAS and HCAP, having done specific integration and testing with both.
In fact, Ocarina can integrate with HNAS in a way that we read a file from HNAS, optimize it, and store the optimized output on HCAP, while the location of the file will still appear to the user to be on HNAS in the same location as always. The block storage behind such a solution could be any HDS Fibre Channel array, from AMS all the way on up through USP-V.
The benefits of tiering and virtualization that you talk about on USP-V are all available to users of such a solution – Ocarina’s optimization and tiering technology is applied at the file level, while USP-V is offering similar optimizations and benefits at the block level. Rather than competing with each other, the two things go hand in hand.
Thanks for listening, and I enjoyed reading this very thoughtful post.
[...] is that dedupe for primary isn’t a panacea. True enough, but as Hu himself has noted in an earlier post, there’s a great advantage to integrating it when you’re already taking advantage of [...]