Hitachi Dynamic Provisioning – Thin, Wide, and Hetero
by Hu Yoshida on Apr 5, 2008
There are multiple dimensions to Hitachi’s implementation of dynamic provisioning. Hitachi Dynamic Provisioning, HDP, is software that runs on the USP V and USP VM storage Controllers and provides the ability to dynamically provision storage as it is required from a pool of storage that is wide striped across any number of logical devices or RAID sets. The available capacity in this storage pool is closely monitored and soft and hard alerts are given when a dynamically provisioned volume approaches its maximum allocation. Alerts are also given when the available capacity in the pool approaches predefined high water marks. When these alerts are triggered, additional capacity can be added to the pool or a volume can be on non-disruptively migrated to a larger pool.
Since a common storage management practice in open systems is to over allocate more storage than is expected to be used, HDP can be used to satisfy the allocation request by thinly provisioning the storage capacity in page stripes of 42 MB as the space is written up to the original allocation request. The rest of the allocated unused space can be used by other applications that share the dynamically provisioned pool. Some file systems and data bases that do a “hard” format when the space is allocated will not benefit from thin provisioning. However they can benefit from other aspects of dynamic provisioning, like the ability to add capacity dynamically to the pool instead of having to install and format another RAID set. Hitachi Data Systems has a number of best practices white papers on HDP, including best practices for Oracle and Windows at http://www.hds.com/products/storage-software/hitachi-dynamic-provisioning.html
Since HDP runs on a massively scalable enterprise storage system, it can be combined with other storage services like point in time copies (ShadowImage), dynamic moves (Tiered Storage Manager), Copy on Write, synchronous replication (TrueCopy), and Asynchronous replication (Hitachi Universal Replicator). The waste of allocated unused capacity is compounded many times over due to the many copies of that capacity that must be generated for: backup cycles, business continuance, development/test, data distribution, Extract/Transform/Load for data mining. HDP can eliminate that additional waste by copying or moving only the dynamically provisioned pages of the volume and not the allocated unused portion of the volume. HDP can copy or move dynamically provisioned volumes (dpvol) in one pool to another dpvol in the same or different HDP pool. It can also copy or move to a regular fixed volume in which case the volume will be fixed from then on. If a regular volume is copied or moved to a dpvol, the entire volume will be allocated in the HDP pool since the USP V/VM has no way of knowing how much of the allocation is actually used.
Even if a volume may not be able to use thin provisioning, volumes can still benefit from the wide stripe pages in the HDP pool. The basic element of dynamic provisioning is a 42 MB page that is striped across all the spindles in the pool. This wide striping improves throughput and response time by parallelizing the I/O across all the spindles in the pool. The Enterprise Strategy Group did an evaluation of HDP’s wide striping performance using a random read/write work load, comparing I/O against one array group of four spindles (2+2) versus 8 array groups of 32 spindles. The results showed an improvement of 716% more transactions at 15 ms or less and an improvement in response time of 118% at the first data point in the curve.
The last point is that HDP can leverage the storage virtualization capability of the USP V/M and can be extended as a service to heterogeneous storage systems that are attached to the USP V/VM. David Marshal who blogs on virtualization reports that Gartner recommends including at least one storage system that offers thin provisioning on any short list. With the USP V/VM you can install one storage system and enhance your existing FC storage systems with the ability to do dynamic provisioning, thin provisioning, and wide striping.
[...] Unlike other thin provisioning solutions, HDP, or Hitachi Dynamic Provisioning, supports other vendor’s storage that are attached, or virtualized, behind the USP V or VM. Since it is provided as a service on the USP V/VM, it can aggregate thin provisioning with other common storage services provided on this platform. This Services Oriented Approach to HDP not only provides thin provisioning to other vendor’s legacy or lower level storage system through storage virtualization, we can also combine this with other USP V/VM services like snapshot copies, non disruptive moves or migration, distance replication, and Copy on Write and enhance the capabilities of these services. [...]