Data Center Advisors

So my last blog was on IBM’s XIV product, and I must say it did create some controversy. I got a few comments along the lines of “really?” and a few that disputed some of the various facts. I even got a call from an old friend of mine who is now with XIV (or IBM or whatever). But I’m reasonably comfortable with what I said.

Now we have “thestorageanarchist” chiming in and among other things he brings up the notion of “Hitachi Math” as a yardstick to compare what XIV is doing with their performance claims. I had to chuckle at that one.

What is at root here is that we (HDS) periodically claim that we can do millions of IOPS (4.5, to be exact, or close to ballpark-ish). But before I go further into this, I should explain what the number is.

Firstly, there is a test that all us vendors do (from an engineering perspective). And if you delve into each vendor’s website you’ll generally find this number of max-IOPS. It is a simple test of reading the same 512-byte block from cache over and over again. No disk drive is ever harmed in this test. Really. But we don’t just do it for fun; it has its purpose. We model in advance how fast the front end can perform and this test just validates that model. It also gives us information in how to architect the array (bandwidth, processors, etc.). It’s kinda like putting a car on the rack to see how fast you can make the wheels spin.

So it’s not just a number to get bragging rights. But like many things engineering, stuff does make it into marketing material. And it actually does give you a relative and very rough comparison of how an array might scale. As in a 1-million IOPS array might just scale better than one with half the IOPS. But not necessarily, which is why we’ve been slogging through TPC and SPC benchmarks for years, since they are more representative of real workloads. So “Hitachi Math” or not, it’s pretty standard. We publish ours because it’s higher than anyone else’s. Do you blame us?

So what about SPC benchmarks? Yes, they are valid and do give you a much better understanding of array performance, and so that you don’t have to slop through a 467-page document, the final SPC-1 number for the USP-V is 200,245.73 IOPS and if someone can explain to me what a .73 IO is I’ll be eternally grateful. “Hitachi Math” at its finest!! Or how about calling it “EMC Math” since they won’t join the SPC. So all of their numbers are theoretical!! I’m still waiting for an explanation (other than the obvious one) as to why EMC won’t join. But I digress……

What is new here is learning that XIV might be spoofing iometer to obtain false results. It’s certainly not a new technique and I’ve seen a number of network-based virtualization companies a few years ago that were doing the same thing. Not cool at all, if it’s true. I hope IBM reads the referenced blog to either (1) refute it with detail, and/or (2) stop the practice. It really makes us all look bad. Although you have to admit it’s pretty clever!

So, yes, the max-IOPS number is real and can be validated and if someone wants to do that, I would suggest you have way too much time on your hands. In this job market that’s not advisable. But what’s really real is what performance you get on your own  workloads. We all know that, and if you don’t know that then get out of the way of people that do know that. That’s all that matters and the SPC-1 benchmarks are guidelines to that.

Anyway, it should be interesting to see how this story plays out. As a vendor I actually hope the XIV story is not true….but stay tuned!