Virtualization platforms
February 11th, 2006
Mike Linett, the President of ZeroWait Corporation sent the following email to me in regards to my post on Virtualization and commoditization.
"In today’s Blog, I found your last paragraph very interesting, because the low end EMC units and NetApp filers are essentially PC based solutions. Additionally, I have heard that NetApp uses NEC motherboards. Both companies claim to be able to virtualize storage now, or are working on virtualization engines on their PC based platforms. In your view does this mean that EMC and NetApp are going to have to supersede their current PC architecture with something else to accomplish their storage tasks and virtualization tasks? If so, what do you think will take their place?"
I thought that was a good question so I got his permission to answer his question in this post.
Virtualization solutions need to be able to scale and provide a secure multi-tenant environment. In order to do this you need reliability, high availability, connectivity, a lot of globally managed cache, and the ability to partition resources to provide a secure multi-tenant environment
PC’s need to be clustered to provide reliability. But PC Clusters do not provide availability. A cluster is like having a pair of legs, if you lose one, you’re crippled and you need to fix it before you go back to work.You lose half your resources and you need maintenance windows to recover. They lack ports for connectivity. The IBM SVC has only 4 ports, two for fan in and two for fan out. Disks need a lot of cache up front to mask their mechanical delays. PC’s have limited cache and when they are clustered, they need to update each other’s cache, since they do not share a global cache. PC’s can not partition what limited resources they have. One application can dominate all the resources and starve the other applications of I/O resources. The Lone Sysadmin posted a recent blog that describes the problems of clustering.
A virtualization platform must be built like a high end storage controller containing a large number of processors (a hundred plus processors) sharing a large global cache. Storage controllers already do a lot of the virtualization, mapping LUNs to Parity groups, to physical disks. Storage controllers also provide services like DR and Point in time copies which require a lot of cache. Improvements have to be made to the storage control unit to improve connectivity by virtualizing the physical port to thousands of virtual ports, giving each of these virtual ports their own dedicated address space for secure multi tenancy, and partitioning of cache to ensure QoS, The cache has to be dynamically configurable in order to non disruptively attaché external storage and move data between storage tiers.
Similar concepts have to be applied to processor platforms to support a scalable server virtualization solution
Hu, I must say that I really enjoy reading your blog. There are many interesting thoughts and observations posted here, but unfortunately sometimes they are mixed with pure marketing FUD. May I ask you to do one thing: could you please mark your marketing posts somehow, so that I could skip and not read them? I do understand that this is a corporate blog, and you must follow certain rules etc. But please, it’s so obvious when you’re trying to slay competitors simply because you have to. Have you ever though about running your own personal blog?
–
“IBM SVC has only 4 ports, two for fan in and two for fan out.”
IBM San Volume Controller has 8 ports in minimal 2-node configuration and up to 32 ports in its maximum configuration. These ports are not dedicated to either fan-in or fan-out. In other words, same SVC ports are used for working with storage and hosts at the same time.
“One application can dominate all the resources and starve the other applications of I/O resources.”
Each SVC-managed virtual disk can be configured with its own throttle rate specified either in IOPS or in MBps. This allows to allocate performance resources between different applications that use SVC.
“You lose half your resources and you need maintenance windows to recover.”
When you cluster eight SVC nodes you would NOT need maintenance window to recover unless you lose all EIGHT of them. And let’s not call them PC’s because
1) These appliances are based on xSeries server platform which are a bit different to Dell Optiplex PC which I’m currently using to write this post.
2) There are several hardware modifications like independent hardware watchdog timer, custom FC HBA’s, etc. SVC is NOT a PC.
And I don’t understand this thoughts expressed by “lone sysadmin” about clustering at all. The question whether to cluster or not could be asked 20 years ago or so (I’m a bit too young to tell for sure), nowdays everybody uses clustering. Show me a company that does not use clustering. Even the biggest machines are clustered these days, we all use redundant servers, fabrics etc. This is the simplest and the most affordable way to dramatically improve reliability of any system.
“they need to update each other’s cache, since they do not share a global cache.”
I really like this one! The only reason SVC nodes need to update each other’s cache is to mirror writes. We all know one hardware vendor who did not mirror cache in its arrays, but this is (surprisingly) not HDS!
–
I’m not going to tell you why HDS approach is not the right one, you can ask {EMC, IBM, …} sales people to do this job. I just admire this blog and I don’t want to see lies here.
Hu,
I think we need to go back first and say, the main goal is to design an architecture thats best for the specific application.
Virtualization does mask complexity but masking technical complexity isn’t just at the benfit of IT people.
Your assessment on PC clustering is correct, hardware provides no resiliency and simple clustering isn’t the solution either.
The need is for smart software that is configured to handle hardware failures immediately and define the QoS of resources.
Hardware doesn’t make itself a commodity, the software that can run on it does.
Nathan
Alex, thanks for reading my blog and taking the time to send your comments.
I stand corrected. An SVC consists of 2 nodes each of which have 4 ports so an SVC cluster has 8 ports. And you are right you can cluster 8 of these together and get 64 ports, some number of which connect to servers and some number connect to storage. What if you need to connect thousands of hosts and peta bytes of storage? I don’t think that provides the scalability you need for a virtualization solution.
Its been some time since I looked at the SVC, so I may be out of date. If you need to do a code upgrade to the SVC clusters can you do it non disruptively?
Storage is all about not losing data. Call me old fashion, but if there is a write to volitile memory, I say it must be mirrored for protection. That vendor you reffered to must agree since he added mirror cache to his latest announcement.
Hello Nathan, you are right. It is the architecture, and it must be designed for the application. The thing to remember is that nothing stands still. As applications or business requirements change and morph, hardware and software will need to change so I doubt that either will ever be commoditized.