In doing some browsing recently I ran across Ken Ow-Wing’s blog I ran across this post. He to observes that the V-Max while architected differently is indeed largely a feature oriented copy of the USP-V. I know that EMC hates this point, but hey truth is truth. One of Ken’s extremely interesting points is:
The reminds when when when IBM clustered a bunch of RS 6000s togeather around a moderately performant switch/network, called it an RS 6000 SP2, and placed Oracle Parallel Database on it for the backend database for applicatioins such as SAP R3.Sun created a competitive program to compete with the IBM SP2 with the earliest ancestor of the Sun Enterprise 9000 server, The Sun Enterprise 10000. which went GA 1997. This was Sun first Supercomputer based on the designs acquired when Sun purchased the Business Systems Divsion of Cray Research (the first and foremost SuperComputer company of that era).
You may notice that the Sun Enterprise 9000 is still around and you do not hear much about IBM SP2s. Sun won. I will spare you more details.
I would like to echo his sentiments and add an additional point: BlueGene. My take here is that at the time just before BlueGene massive parallel clusters were all the rage in the US. In fact the Top500 list was dominated by them. Then along came the the NEC Earth Simulator freaking everyone out. If I recall it held the top spot for two years and effectively freaked out the US Congress. The Congress subsequently appropriated funding and basically pushed IBM and Cray to make new systems that punished the Earth Simulator. These systems are more like the system that Ken mentions so indeed Sun’s architecture was proven correct over time.
Another of Ken’s points is regarding HTSM and he is right we’ve had the capability that EMC is talking about with FAST since 2004. It has gotten way better over time, for instance migrating between thick and thinly provisioned LUs, or being able to provision from a tier, or, or, or. I do now have a debate with EMC on the number of concurrent migrations. They are claiming 128 times improvement over Hitachi on migrations. However, let’s start taking a look at some of EMC’s own documentation and ask some simple questions. We can see if they will answer or not.
On page 9 of the document referenced above is the following statement:
Sessions
Migrations are submitted and managed as sessions - a session name is specified when establishing the migration. Symmetrix support up to 16 concurrent active and passive sessions. An active session is defined as a migration that has been established but has not yet completed. A passive session is a migration that has completed, but has not yet been terminated
The Symmetrix supports up to four concurrent active Virtual LUN migrations, although only one migration can be submitted at a given time. Once a migration as been established another migration can then be submitted.
Having been someone who reads patents for a variety of purposes as a part of my job I find it hard to parse and understand the above. Does it mean that only 4 Virtual LUNs can be migrated at a time? How many LUNs are in a session? Reading a little further down in the document is a description of the “SyncInProg” state for migrating Virtual LUNs. So I guess my question is how many “SyncInProg” LUNs supported in parallel? Note I’ve honestly tried to find out by looking at the V-Max data sheet and the spec sheet. All I get is a rather Dr. Evil response of “muh ha ha ha thousands of migrations in parallel” from the EMC folk. So I looking for an honest answer here. How does this work?
Comments (14 )
the storage anarchist on 10 Jul 2009 at 3:47 am
The maximum number of concurrent LUN migrations is 128 LUNs (or CKD volumes) per engine, or 1024 parallel “SyncInProg” LUNs in a fully-configured 8-engine V-Max. These operate concurrently with other copy sessions in the V-Max, so unlike USP-V migrations, V-Max local and remote replication sessions do not need to be paused or suspended during the migrations.
If I understand Hitachi TSM and USP-V documentation correctly, you are limited to a maximum of 64 LUN migrations in the queue, and only 8 are executed concurrently with a 15 minute pause in between batches. As a result, the V-Max executes migrations significantly faster than the USP-V.
V-Max also does not require customers to operate outside of remote replication compliance while these often lengthy migrations are underway; TSM on USP-V requires replication sessions to be suspended for the duration they are in the queue (you cannot add migration requests for LUNs/Volumes with active replication).
Ken Ow-Wing has misrepresented the V-Max bandwidth to be the 10GB/s bandwidth of only a single engine; a full 8-engine V-Max has 8 times that non-blocking bandwidth of a single engine (yes, it scales linearly). You must also consider that a significant percentage of memory accesses will be local and not required to traverse the RapidIO fabric at all.
the storage anarchist on 10 Jul 2009 at 1:25 pm
Copy this: http://thestorageanarchist.typepad.com/weblog/2009/07/2015-challenge-accepted-free-vp.html
Christophe Bertrand on 13 Jul 2009 at 4:36 pm
I think you’ve got a few things wrong Barry, but I will let Michael answer if he hasn’t done so yet. In any case, what prospective customers need to remember is that this conversation is really very theoretical for VMax at this point: how many customers do you really have (I mean paying, not “seeded”)? We’ve got a proven architecture and many customers who derive great operational benefits today.
the storage anarchist on 14 Jul 2009 at 3:11 am
Umm…I only mentioned ONE thing, Christophe:
Symmetrix Virtual Provisioning is now FREE. No artificial limits. No “new customers only.” Totally free, for new and existing V-Max, DMX4 and DMX3 arrays.
Improved utilization. Streamlined storage allocation. Inherent wide striping.
All for $0.00.
I’m sure your customers and propsects alike would love to hear what you think is wrong with that. Please, do tell…
Christophe Bertrand » Blog Archive » Is VMax stuck on the runway? on 17 Jul 2009 at 3:37 pm
[...] enough, Mr. Burke from EMC avoided the question when I asked him directly on Michael Hay’s blog, essentially confirming that people are just not [...]
the storage anarchist on 22 Jul 2009 at 3:57 am
Christophe -
As I responded in the comment I left on your blog (which you apparently chose to censor), we are in the midst of EMC’s quarterly earnings Quiet Period, and thus NOBODY from EMC can comment on revenues or unit sales.
When EMC chooses to announce relevant specifics, I’ll be most happy to discuss them. Until then, I must be QUIET!!!
Meanwhile, you’ve not answered MY question: what is Hitachi’s response to FREE Virtual Provisioning on V-Max and DMX?
Vinod Subramaniam on 23 Jul 2009 at 10:33 am
I think we need to give EMC credit for recognizing that in most monolithic subsystems the backplane has become the limiting factor in scalability.
What EMC has achieved is a Virtual Backplane based on the RapidIO interconnect. Interestingly enough Storage Technology Corporation holds US Patent 7,002,961 on Virtual Backplanes.
http://www.patentstorm.us/patents/pdfs/patent_id/7002961.html
I think HDS needs to let customers know what they are doing in terms of
A. Grid Storage Architectures
B. Backplane Interconnects
– Vinod
Michael Hay on 24 Jul 2009 at 4:22 pm
Vinod, what I’ve been trying to point out that I don’t think anyone is yet willing to hear is that the super computing industry had the same lines of thinking for a long time: build really huge compute farms using some kind of a interconnect (you call it a virtual backplane) and we can solve world hunger. For a while indeed this seemed to work out and was originally innovated out of NASA (although I’m sure someone can show me an earlier example if they dare). What happened, as I point out in this post, is that someone, NEC, rethought the rule that Linux-MPP was the answer to everything and smashed the records for two years. The US computer makers and the US Congress were freaked out and out of this came funding and eventually BlueGene/L which smashed the Earth Simulator. I personally think that this is important because when the Super Computing industry does something it is 5-7 years ahead of the business computing industry and now we are seeing the business computing industry beginning to adopt BlueGene technology, FPGAs, ASICs, GPUs, etc. to gain a competitive advantage in their field where processing speed matters. Does anyone ask IBM what interconnect they are using for BlueGene, or the p595, or even the SP2 back in the day? Nope they don’t, basically no one cares. Sure there is the odd competitor but that is because they are trying to ding a successful platform. What the customers care about are fast processing times and getting their jobs done on time. So what I really think is that EMC is glomming onto an interconnect or virtual backplane that in the long run no one will really care about, because the Symmetrix product line has been largely devoid of innovation for a long time. I mean if you have to market infrastructure-ware as something innovative and novel that speaks volumes.
As to the storage grid I have a simple answer to that question: UVM and virtualization along with HAM. We have it we’ve been working it since the introduction of UVM with the USP in 2004/2005. In essence we are able to use an existing interconnect, Fibre Channel, and allow activities like RAID rebuild, shredding, migrations, etc. to all occur either on each node independently or controlled by the master node. Further to maximize the processor utilization within all USP-V’s we distribute jobs to idle processors so that we can get more performance out of the hardware without forcing our customers to continually have to buy a new brick. If you want to talk about the cartesian scaling of HNAS which sports iSCSI and NFS/CIFS then there we can scale to eight (8) nodes within the single namespace and pretty much smash the competition.
I’ll end my response by quoting my friend Shmuel Shottan: every year I ask my engineers to investigate if they can replace the NIM with an industry standard TOE, and every year they come back and say nope. This is due to the differentiation that BlueArc has within their NIM complex versus the state of the art commodity component. What I’m getting at is that there is a cost benefit analysis which needs to be done to adopt new technologies. Simply doing it for fun is not sound engineering. I’ll say this again, if EMC is marketing infrastructure-ware to the masses then heck. Basically I see Hitachi as Apple and EMC as Microsoft, but in the storage market. Everyone knows that Microsoft copies Apple and it is okay. I think EMC should stop trying to counter the argument and just move on, it is truth and there is no shame in it!
Michael Hay on 24 Jul 2009 at 4:30 pm
Barry here is my response to your inaccuracies about HTSM and USP-VM.
http://blogs.hds.com/michael/2009/07/my-response-to-barry.html
Vinod Subramaniam on 25 Jul 2009 at 2:47 am
Michael
I’ll leave the analogy with supercomputing alone. The analogy itself is good but I dont like projections or predictions purely based on analogies.
Let me deal with the backplane interconnect or backplane scalability issue. A few facts before I do that.
7700E - Internal Bandwidth 1.1GB/sec - Gen 0
9900 - Internal Bandwidth 6.5GB/sec - Gen I
9900V - Internal Bandwidth 15.6GB/sec - Gen II
USP - Internal Bandwidth 80GB/sec - Gen III
USPV - Internal Bandwidth 106GB/sec - Gen IV
Fact A :- It is hard to push 10Gbps over copper traces on a PCB over 3 ft owing to high frequency attenuation and strong electromagnetic inter-channel crosstalk. Though there have been research papers published proving that one can push upto 20Gbps over copper traces on a single lane.
Fact B :- Gen II HiStar doubled the aggregate internal bandwidth over Gen I. Gen III more than quadrupled the aggregate internal badnwidth over Gen II. However the USPV internal data bandwidth is the same as the USP. Is this because of Fact A above ?
Router vendors are working with Optical Backplanes with optical pathways instead of copper traces and are also looking at optical interconnects for line cards. Hence my question on how Hitachi plans to bump up the internal bandwidth beyond what the USPV can do today ?
– Vinod
Michael Hay » Blog Archive » My Response to Barry - UPDATED on 25 Jul 2009 at 5:02 pm
[...] and readability of this post. To do that I’m posting Barry’s original comments on this post as a PNG here. I did that because I want to be clear on not censoring since Barry is so sensitive [...]
Vinod Subramaniam on 29 Jul 2009 at 12:28 pm
Michael
A little more digging revealed the answer. Here’s a paper from IEEE with Shinji Nishimura of Hitachi Ltd as a co-author.
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04604617
Here’s a quote from the paper :-
“Optical Backplane :- It is expected that a maximum throughput of
around 10Tbps will be required for not only core routers and core switches but also edge routers and edge switches, by year 2010. As it is difficult to handle such large traffic under reasonable cost and power consumption only by electrical devices, we believe optical technology will be needed. ”
So how long before this technology enters Hitachi Storage Arrays.
And you also mentioned FPGA’s in a earlier post. That makes me wonder what Hitachi is doing with reconfigurable computing.
Right now the way partitioning works on the USPV there is no way to use separate algorithms for each partition based on the workload. I guess that is why the Write Pending watermark of 40% is global with a SOM to set it to Average Write Pending instead of Max Write Pending. So what if FPGA’s were dedicated per partition dynamically and the algorithms programmed differently per partition based on the workload ?
– Vinod
Michael Hay on 29 Jul 2009 at 5:45 pm
Vinod, unfortunately due to my position I am not able to talk about these kinds of forward looking things publicly. I will say that Hitachi continues using the R&D muscle that it has across a broad variety of technology areas to meet customer challenges now and in the future. This is and continues to be one of Hitachi’s strengths. For instance our ability to water cool a processor directly comes from nuclear power cooling research and was applied to our mainframes in the past. Our competitors and the press tend to say that having broad research focus is a bad thing. I totally disagree with this chain of logic. As we all know the market longs for these fundamental research organizations, for example Bell Labs is often lusted after even though it no longer exists. Well Hitachi, IBM and others still have this fundamental R&D capability and not only does it further commercial ventures, it also contributes to the well being of the human condition.
Bill Bartmann on 01 Sep 2009 at 11:47 am
Cool site, love the info.