Hu's Blog

IDC published a report this month entitled, “The real Costs to Power and Cool All the World’s External Storage”.  It’s a good report which I would encourage you to get for an understanding of the relationships between, capacity, performance, power, cooling, and cost. IDC has been able to extract their findings from their comprehensive collection of data on shipments of external disk storage since 1998. By the end of this year, 2008, IDC is predicting that there will be:

An install base of 49 Billion HDDs

Consuming 22 BkWh for power and cooling

At a total cost of $1.76B

Producing 34.1 B pounds of CO₂

While they see storage capacity growing by 50 to 55% per year, they see power consumption growing around 10% per year due to increasing HDD capacity and other strategies to reduce power costs associated with external storage.

While they enumerate these strategies they caution that customers must be careful in deciding which technologies best fit their needs. IDC points out that green technologies like spin down or larger capacity disks may impact performance and run counter to their other business goals. They also encourage vendors to offer complimentary strategies to help customers reap the full benefit of “turning green”. Rather than list their green strategies here, I will leave it to you to obtain this paper from IDC.

However, one of the strategies that is not on their list is Dynamic Provisioning. Hitachi Dynamic Provisioning has the ability to dramatically increase the utilization of storage by eliminating the waste of allocated but unused capacity and increase the performance of larger capacity, power efficient, disks.  This is done by striping pages across the width of a pool of disk RAID array groups. This wide striping enlists more disks in servicing an I/O request which increases performance. These pages are allocated dynamically as a volume is written to, rather than allocating the entire volume on one RAID array group at the beginning. This strategy has many complimentary benefits: increased utilization of storage capacity; increased performance through the use of wide striping, and reduced operational costs with simpler provisioning and performance tuning.

For example, let’s say that you have your volumes are distributed to 10 RAID array groups and have only 20% utilization. With Dynamic Provisioning we could combine 4 of these RAID array groups into a common pool of storage where the volume’s capacity is allocated from pages that are striped across all the disks of the 4 RAID array groups. Since we no longer have silos of RAID array groups we can now get better utilization of the capacity and we only allocate the capacity as it is actually used. Also since we now have volumes that span 4 RAID array groups, we have 4 times as many arms processing the I/O. We can release the other 6 RAID Array Groups and more than halve our power and cooling requirement just by going from 20% to 50% utilization.  

This strategy for dynamic provisioning is considered a common reusable service on the Hitachi Universal Storage Platform. V/VM. It can be combined with other services that are available on that platform, such as copy, move, and replication services. It can also be combined with virtualization of external storage so that dynamic provisioning can be extended to enhance the utilization and performance of lower level, external, storage.

The combination of Hitachi Dynamic Provisioning with virtualization of external storage can increase utilization of storage from around 20-30% to 60-80% and dramatically reduce the total power per disk which IDC estimates today to be 48 w (12 w per disk, 12 w for storage infrastructure, and 24 w for cooling).