Tiering with SATA or Capacity Enhanced SAS
by Hu Yoshida on May 23, 2011
I seemed to have touched a nerve with Barry Burke in my last post on tiered storage myths, where I talked about the advantages of tiering across internal and external tiers of storage. Since EMC is not able to provide this capability Barry went on a long rant on why he thinks Hitachi Data Systems supports external tiers of storage with our virtualization approach.
Barry and others post such rants to provoke direct responses, which inevitably leads to a vicious cycle of comments that customers generally frown upon. I blog first and foremost for my customers. When I wrote that tiering within a single storage frame was not the most economical way to tier storage, Barry tried to switch the conversation to the Read after Write that we use with SATA disks to guard against silent corruption. Apparently he does not think that such precautions need to be taken with SATA disk.
Lower tiers are for Capacity Optimization
First of all, whether we tier internally or externally is not an either or decision. We offer external tiers of storage where it makes sense for our customers. We do not force our customers to buy another tier 1 storage frame to get a lower tier of storage, especially if there are existing storage frames that can fill that need. Lower tiers of storage are there for capacity optimization and not for performance, so SATA as a lower tier of storage is a good solution as long as we can ensure the integrity of the data recording. Whether it resides internal or external to VSP, it does not impact the access to the higher performance tiers of storage. Unlike the VMAX architecture where one processor supports all the front end, back end, and tiering workload, VSP has separate processors that distribute the different workloads.
Why does Hitachi do Read after Write with SATA?
While SATA, SAS, and FC disks have about the same meantime between failure rates, SATA disks lack the addressability to provide end to end, initiator to target, nexus checking ( IOECC/IOEDC), which insures integrity. Silent corruption can occur with SATA and so Hitachi uses read after write to detect and correct this corruption. This does add another rotation and does add a performance overhead. In order to reduce this overhead in VSP, we also offer the option of SATA-E, which generates a write checksum block. This eliminates the Read after Write and provides improved performance while insuring write integrity on the SATA disks.
Advantages of Capacity Enhanced SAS
Today, we have a better solution for capacity tiers with the availability of capacity enhanced SAS (Serial Attached SCSI) disks. We offer a 3.5 inch, 7200 rpm, SAS disk with a 2 TB capacity on AMS 2000. This disk is about the same cost as SATA, but unlike SATA it has the end to end error correction and error detection codes, which ensures that data traveling to and from the SAS disk is not misdirected, so this helps with the integrity of reads as well as writes. In addition, SAS has dual port redundancy and point to point connectivity which ensures better system availability and fault isolation than SATA. The availability of these enhanced capacity SAS disks on VSP is awaiting the completion of qualification tests. With the availability of capacity enhanced SAS at a comparable price, there is no longer any need for SATA.
So who still needs SATA?
Capacity enhanced SAS is only available for storage systems with a SAS backplane, like VSP and AMS 2000. Other systems that use FC back planes still need SATA for capacity optimized tiers, unless you have FC USP and USP V that can attach SAS enabled storage systems like AMS 2000 as external tiers of storage. Competitive FC storage systems that cannot attach internal or external SAS storage will be limited to SATA with the integrity concerns that are very real with SATA.
What do we call it?
I’d like to make one additional note on capacity enhanced SAS disks. Some of the literature on this topic refer to this as “nearline” SAS. This is a misnomer since this implies that this type of SAS is not online and immediately accessible. This type of SAS is online. It rotates a little slower so that we can pack more bits on a disk but there is nothing nearline about this disk. Calling it “Fat” SAS is also not very respectful of the benefits of this type of drive. However, it does take time to say “capacity enhanced” or “large capacity” SAS every time I want to refer to it.
If you have any ideas for naming this class of disk so that its characteristics could be easily recognized, please post a comment with your thoughts.
Comments (7 )
Actually, Hu, my purpose was to point out to our collective customers and prospects some facts that you had neglected to mention in your article. Perhaps you didn’t think them pertinent, but your customers deserve to be told the whole truth, not just the part you think interesting.
My comments therefore pointed out:
1) that tiering inside the array is by definition more cost-effective than using external capacity, at least if you use the same type, size and number of drives in either case.
2) that write/read/verify does not protect against silent data corruption at all, since a successful verify now does not guarantee that a future read request won’t be corrupted or mis-read.
3) that the performance implications of write/read/verify is significant, which you now seem to have conceded.
VMAX has used an adaptation of T10DIF on all drives it supports since its introduction in April 2009, including SATA. By all appearances, SATA-E is an attempt to mimic this capability. I notice that SATA-E is supported by HDT for internal capacity only as of a code update for the VSP that was released at the beginning of May 2012.
I have to disagree with your assertion that arrays that use an FC back-end are limited to using only SATA drives. As I am sure you know, supporting SATA on an FC back-end requires a “bridge” to convert the interconnection and the protocol – in fact Hitachi used just such a bridge on the USP-V, and EMC uses one on VMAX. Your readers should not be surprised to learn that such a bridge between FC and SAS interfaces is also technically feasible.
No matter how you slice it, if you can add another tier from an existing frame that you already have, it is cheaper than buying a new tier 1 frame, and maybe another VMAX controller to get your tier 3 storage. There is no argument about SATA having slower performance. A write verify insures that the data is written correctly. Capacity enhanced SAS is a better solution. If you adapt a SAS drive back to a FC loop bus, it negates the benefits of point to point SAS.
Cool – PechaKucha style comments!
“add another tier from an existing frame” – doesn’t explain why HDS sales reps will nearly always quote a NEW AMS behind a VSP for tier 3, instead of putting the tier 3 drives directly into the VSP. And an “old” array typically has a very significant maintenance cost overead, no matter who the manufacturer is.
“Negates the benefits of point to point SAS” – what benefits? I must have missed that post. I can guess you’re going to make some argument about arbitrated loop vs SAS switches…before you do, recall that VMAX and CLARiiON both use switched FC for their FC back-end. And please don’t start trying to convince people that 6Gbps SAS is measurably faster than 4Gbps FC – on both SSD and HDD, the transfer time is a infinitesimal portion of the total I/O latency.
Write verify only ensures the correct data was written (somewhere), but it does not (and cannot) ensure that the proper data is read later – if it did, you wouldn’t have implemented SATA-E. So why did HDT require write-verify for the first 6 months it was shipping (and why does the AMS still only support this mode on SATA)? Does Hitachi intentionally expose its SATA customers to silent data corruption, even after you know that SATA-E offers better protection?
If I can clarify; with regards to SATA, Barry is stating that write/verify is insuring against the wrong type of SATA error. He’s not arguing that the data being read is incorrect versus what was written, or that write/verify won’t protect against that type of error. He is stating the fact that the majority of errors from SATA come from the data being read from the wrong location. So write/verify is not providing the proper protection against the #1 error of SATA drives. You’re insuring against tornadoes when the real problem is earthquakes.
I find it amusing you wrote:
“Barry and others post such rants to provoke direct responses, which inevitably leads to a vicious cycle of comments that customers generally frown upon”
And guess what? You got exactly that!
Maybe its best to just avoid mentioning certain people by name? #;-)
so what do we call the capacity enhanced SAS HDD’s? Well, they are online, not nearline, and they are certainly lower cost devices compared to SAS. Therefore I would call them LC-SAS (lower cost SAS).
so what do we call the capacity enhanced SAS HDD’s?
Since the devices provide the checking for data authenticity, how about
CHECK-SAS or more briefly, CK-SAS…