There’s been practically a firestorm when EMC announced ViPR, its own version of “software-defined storage” at EMC World last week. Whether you want to call it Virtualization Platform Re-defined or Re-imagined, competitors such as NetApp, HDS, Nexenta have taken pot-shots at EMC, and touting their own version of software-defined storage.
In the release announcement, EMC claimed the following (a cut-&-paste from the announcement):
- The EMC ViPR Software-Defined Storage Platform uniquely provides the ability to both manage storage infrastructure (Control Plane) and the data residing within that infrastructure (Data Plane).
- The EMC ViPR Controller leverages existing storage infrastructures for traditional workloads, but provisions new ViPR Object Data Services (with access via Amazon S3 or HDFS APIs) for next-generation workloads. ViPR Object Data Services integrate with OpenStack via Swift and can be run against enterprise or commodity storage.
- EMC ViPR integrates tightly with VMware’s Software Defined Data Center through industry standard APIs and interoperates with Microsoft and OpenStack.
The separation of the Control Plane and the Data Plane of the ViPR allows the abstraction of 2 main layers.
Layer 1 is the abstraction of the underlying storage hardware infrastructure. Although I don’t have the full details (EMC guys please enlighten me, please!), I believe storage administrator no longer need to carve out LUNs from RAID groups or Storage Pools, striped and sliced them and further provision them into meta file systems before they are exported or shared through NAS protocols. I am , of course, quoting the underlying provisioning architecture of Celerra, which can be quite complex. Anyone who has done manual provisioning with Celerra Manager should know what I mean.
Here’s the provisioning architecture of Celerra:
Even though NetApp’s ONTAP and WAFL provisioning architecture is more simplified than the original Celerra provisioning architecture, and EMC VNX Unisphere has un-obfuscate the file provisioning process in unified VNX, fundamentally, the process and the science behind storage capacity provisioning have not changed.
EMC ViPR Controller, which is the Control Plane, now looks at the storage components and resources as a consolidated resource repository (I am resisting to call it a pool for obvious reasons). The beauty of this approach is that a storage administrator can now call the shots of storage provisioning that is more congruent with business and operational requirements.
I believe one can say -“Give me 10GB of capacity that will continue to serve an requirement of 5% growth rate and is able to meet an RPO of 8 hours. Meanwhile the data in the provisioned capacity should meet an expected SLA of a defined level metrics, and has priorities for certain transactional requirements and yet maintain compliance and recovery throughout a certain geographic region”. And all the storage administrator has to do is to create a job policy that will provision the storage, and manage the underlying storage and data infrastructure throughout the lifecycle of the storage container and capacity as it is used.
Or course, I could be completely wrong with what I just wrote in the above, and I am hoping that someone with deep knowledge of the EMC ViPR controller’s administration to give me lesson of storage admin that I would never forget.
But still, that is one heck of a wish. It’s like Tony Stark commanding Jarvis in the most sarcastic and yet subtle manner, and Jarvis is still able to pull it off excellently. Well, I do hope that the EMC ViPR controller job policy engine could evolve into what Jarvis can do for Ironman and Tony Stark. 😉
And it would be super cool to have a EMC ViPR dashboard interface like Jarvis. See below:
The other layer is the Data Plane and this is something that is extremely interesting because it would be able to switch between traditional workload and “next-generation” workload. If the workload is for SAN or NAS, i.e. blocks and files respectively, ViPR does not take an active role and basically passes off the data plane function to the underlying storage array controller.
However, for “next-generation” workload that involves hyperscaling of storage resources for cloud computing and big data, those that involves data of the 3Vs (volume, variety and velocity) nature, the Data Plane takes the data and objectify them into object-based data objects and distributing them with HDFS (Hadoop Distributed File System), across platforms like OpenStack Swift and Amazon S3. At the same time, newer access methods such as REST and HDFS are integrated as part of the “next-generation” data object access and delivery mechanism. The HDFS Data Service also provide in-place analytics. In all, this is the EMC ViPR Data Object Services.
EMC has announced that the initial ViPR will support EMC Atmos, EMC VNX and EMC Isilon as the persistent layer of storage infrastructure and also commodity hardware (for OpenStack Swift and Amazon S3). I even read somewhere that NetApp FAS and E-Series are supported as well, but don’t quote me for that.
The idea of having policy-based provisioning is not new. I have seen from time and time again, many 3rd party storage software solutions meaning to do just that.
One that comes to mind is the vendor-agnostic DataCore SANSymphony solution and they have been doing it for years. Another one that is closer to the heart is a little known company called Menloware. I personally know the creator, and founder of Menloware and I would suppose he would be happy if I did a little advertising for him.
Below is the architecture diagram of the policy-based storage management software:
Note: If you guys are interested in Menloware, a policy-based storage management software which is vendor-agnostic, let me know. (Brad, this is for you!)
I am not going to decide if this “Software-defined Storage”(SDS) thingy warrants any debate. I will let the analysts and punters do that. The satisfying thing for me was to read the type of spin and BS everyone has on SDS, and trying to bash EMC’s version of SDS. It is really funny when you have this “pot calling the kettle back” situation.
Read this article to see how hard NetApp is trying to convince everyone about them being hardware independent. Or this article on several EMC competitors’ take on SDS.
This is the one that HDS encounters epic failure when their CTO, Hu Yoshida, said in a different article, that “people are buying dumb storage“. How can you say you are software-defined storage when your CTO is bashing customers buying dumb storage? If it is SDS, then obviously the intelligence is in the software but apparently their CTO wants people to buy intelligent storage hardware. HDS has to make up their mind where they want their storage intelligence to be – software or hardware?
I am not saying what EMC is doing is the correct way of doing SDS, but when one starts to decouple the storage hardware and storage software, customers will have a choice to decide what type of storage hardware they want to use, especially with commodity hardware.
It’s been too long that customers are pulled through their nose rings and locked in by proprietary storage hardware. EMC ViPR can be a turning point. I think the storage vendors owe the customer that.
