Hitachi Dynamic Provisioning: It’s a lot more than Thin Provisioning
by Claus Mikkelsen on July 20, 2009
I’ve been on this “Mission from God” (with all due respect to Messrs Aykroyd and Belushi, Blues Brothers, circa 1980) over the past few months to promote the additional benefits of HDP.
Check out the above video where I explain in a bit more detail the benefits of Dynamic Provisioning outside of the more well-known ‘thin provisioning’ benefits. For those who don’t have perfect streaming, or don’t have the time to watch my 3:30 minute video, here’s what I cover:
- There’s a significant performance boost from the wide striping of large page sizes, since it essentially allows a high number of drives to participate in read activity.
- Because of the wide striping, we also have essentially obviated the need for what I call “provisioning for performance” since HDSP will out-perform anything that is manually provisioned
- Carve out what you need, not what best fits the storage array to maximize capacity. Since LUNs are carved out of a common pool, anything to do with physical awareness of LUNs - for example static LUN sizes - are a thing of the past. You need 3GB? Just provision it. How about 147GB? Provision that as well. A 1.3TB LUN? You get the idea.
- What we’ve done is essentially provided that last abstraction layer of virtualization so that we can all start provisioning storage based on what the application needs. That’s a significant milestone.
Hope you enjoy! Look forward to your thoughts.
Comments (9 )
Nigel on 22 Jul 2009 at 3:27 pm
Claus,
Is there a prize for configuring a USP V with classical LDEVs that outperforms one with HDP LUNs?
Oh and nice video, but you forgot the cows in the background -
http://blogs.rupturedmonkey.com/?p=461
http://www.storagerap.com/2009/06/hds-catastrophic-storage-management.html
Nigel
Claus on 22 Jul 2009 at 10:34 pm
Hi Nigel, Yeah, of course there is a prize! I’ll figure that one out. How much time do I have? As far as the cow scene, actually we did make the original film in a local cow pasture, but started receiving legal challenges on trademark infringement from Ben and Jerry’s Ice Cream. So we decided to moo-ve it indoors.
Nigel on 23 Jul 2009 at 3:08 pm
So if you’re serious about this challenge to beat an HDP config….. what are the ground rules and the constraints, what is the workload type and config to work with - you need to be more specific.
BTW if the challenge was more tongue in cheek then Im OK with that and wont moan about it. But if you really have laid down the gauntlet then more specifics are needed.
I love a technical challenge.
Claus on 24 Jul 2009 at 11:12 am
“Tongue-in-cheek” is a better description. Anything else means I’d have to define the workload and parameters, define the configuration, talk to lawyers (ugh!), go down to the trophy shop and have something engraved, you get the idea. I mean, it took the SPC how many years before they could go “live” with their benchmarks?
No, it’s more of an intellectual challenge to get folks to understand the concepts of large-page wide striping, especially for cache-unfriendly workloads. When I talk to DBA’s about this, they “get it”. It’s all about how many drives you can have participating in those nasty read commands, right?
So yes, if you look closely you might have noticed that bulge in my right cheek when I said that.
Hu Yoshida » Blog Archive » Chunk size matters on 30 Jul 2009 at 8:53 pm
[...] is that the smaller the chunklet size, the more efficient the thin provisioning. However, thin provisioning has many more benefits that have to do with ease of provisioning, space reclamation, and data dispersion (wide striping) [...]
SR on 06 Aug 2009 at 12:55 pm
Because of the wide striping, we also have essentially obviated the need for what I call “provisioning for performance” since HDSP will out-perform anything that is manually provisioned
- Carve out what you need, not what best fits the storage array to maximize capacity. Since LUNs are carved out of a common pool, anything to do with physical awareness of LUNs - for example static LUN sizes - are a thing of the past. You need 3GB? Just provision it. How about 147GB? Provision that as well. A 1.3TB LUN? You get the idea.
- What we’ve done is essentially provided that last abstraction layer of virtualization so that we can all start provisioning storage based on what the application needs. That’s a significant milestone.
I’ve been screaming these exact things for the past 6-8 months. Now if only I could somehow buy all this storage at a standard and cheaper (same $/tb each time) rate it would make my managers back down off the NetApp totem poles!
Vinod Ganesan on 23 Sep 2009 at 3:34 am
Claus,
I am a big fan of HDS technology & i am sure the Dynamic provisioning is a great feature that will benefit customers. However, i had a very specific question from a deployment standpoint considering that all of this is being managed from the storage layer now. If i have provisioned a single 3TB ( striping across 128 spindles for performance ) what is the SSD_THROTTLE_MAX value that i need to set in the Solaris 10 operating system? In absence of a recommended number, we will get into serious performance issues & the whole concept can leave a bad taste.
Will eagerly wait for your response.
Claus on 30 Sep 2009 at 6:31 pm
Vinod, Sorry for the delay in posting, but the information you are looking for is in the “Architecture and Concepts Guide USP V”. If you forward me your private email address I can send it to you, along with some more detailed information of your question from our performance group. Good stuff! And thanx for the question…
HDS and VAAI Integration » boche.net – VMware Virtualization Evangelist on 03 Apr 2011 at 4:10 pm
[...] Hitachi Dynamic Provisioning (HDP) technology which is essentially HDS thin provisioning plus other lesser known benefits but has nothing more to do with VAAI than any other storage vendor which supports both VAAI and [...]
