And Cloud Storage will make us even stranger

It was a dark and stormy night ….

I was in a car with my host in the stifling traffic jams on the streets of Jakarta. We had just finished dinner and his driver was taking me back to the hotel. It was about 9pm and we were making conversation trying to figure out how we can work together. My host, a wonderful Singaporean who has been residing in Jakarta for more than a decade and a half, owns a distributorship focusing mainly on IT security solutions. He had invited me over to Jakarta to give a talk on Cloud Storage at the Indonesia CIO Network event on January 9th 2013.

I was there to represent SNIA South Asia to give a talk about CDMI (Cloud Data Management Interface), and my host also took the opportunity to introduce Nutanix, a SAN-less 2-tier, high-performance, virtualized data center platform. (Note: That’s quite a mouthful, but gotta include all the buzz-words in there). It was my host’s first foray into storage networking solutions, away from his usual security solutions spread. As the conversation went on in the car, he said “You storage guys are so strange!“.

To many of the IT folks who have been involved in OS, applications, security, and networking, to say a few, storage is like a dark art, some mumbo jumbo, voodoo-like science known to a select few. That’s great, because this perception will keep us relevant, and still have the value and a job. To me, that just fine and dandy, and I like it that way. 🙂

In preparation to the event, I have to learn up SNIA CDMI. Cloud and Storage … Cloud and Storage … Cloud and Storage. Hmmm ….

Cloud Computing have different meanings to different people. To me, that cloud concept is still very much evolving and when it is coupled with the strange world of storage networking, it becomes even “cloudier” (pun intended) and stranger. So this post is about putting some cloud storage concepts into perspective, the SNIA way. And I would like to share that experience with the readers.

SNIA is not always everyone’s cup of tea. I know that because I speak to many people about SNIA. Of course, naysayers are abound and whenever I talked about SNIA, the defensive walls come up. I recalled a few weeks ago, I mentioned SNIA CDMI to some folks in the EMC IT-as-a-Service class I attended in Singapore. One particular IT manager from a renowned Singapore government-linked agency was so eager to prove me wrong, saying that standards and APIs like SNIA are only as good as the adoption by the industry players and if they don’t adopt SNIA CDMI as part of their offering, it will be the death knell of the cloud storage standard.

Yes, I agree with him about the adoption bit, because the Cloud Storage industry has to grow with the support of industry players. But don’t be so fearful of standards like CDMI. The intention is to grow the cloud storage market, and have it stabilize and mature so that everyone can benefit from it. This everyone is including Cloud Storage vendors, storage vendors, storage consumers, advocates of storage standards, Cloud Computing vendors and other industry supports such as DMTF (Distributed Management Task Force) and Cloud Security Alliance. In the end, the objective is to make sure everyone can benefit from the implementation of standards such as CDMI.

I was researching the subject cloud storage and found an article dated in February 2012 that listed 97 cloud-related APIs. Ninety seven APIs! And out of that lot, there 16 (sixteen) cloud storage-related APIs! That means LOCK-IN! Not good for the industry, dude!

What is CDMI (Cloud Data Management Interface)? CDMI is SNIA’s attempt to introduce inter-operability, portability and consistency in the cloud storage industry. SNIA uses its unique position as a industry voice to promote the CDMI standard and fundamentally, encouraging the innovation and commercialization of a new Data-Storage-as-a-Service industry. Of course, there are other competing cloud storage standards such as Amazon S3 and OpenStack Swift. Objectively, CDMI also has specifications to work with these competing Amazon S3, OpenStack Swift, and other more traditional file access methods such as WebDAV, and NAS protocols like CIFS and NFS. In addition, SNIA is working CDMI to integrate (in ways I do not understand yet) with other standards such as DMTF’s OVF (Open Virtualization Format) and CIMI (Cloud Infrastructure Management Interface) to accelerate CDMI’s adoption.

CDMI has specifications for client-interface to access the cloud storage objects and managing these objects.  CDMI works with cloud storage objects and containers to abstract the underlying storage infrastructure. It has an “export” function that enables various protocols to access the CDMI containers. But the most important thing, in my opinion, is CDMI is also about advanced data services. It standardizes much of the metadata defined of the cloud storage objects and from the metadata, vendors can create new data services such data policies, archiving and retention, data mobility and movement services. These advanced data services, as SNIA hopes, will help create new opportunities and transform the Data-Storage-as-a-Service (probably Cloud Storage is the pre-cursor) industry from the Dropbox-like mindsets to greater heights. One that truly meant for businesses.

A good view of what SNIA CDMI is like is from a diagram I got from David Slik’s blog. Here’s how it looks like from a high level view:

Incidentally, David Slik, a Technical Director of NetApp is also the co-Chair of the SNIA Cloud Storage Technical Working Group (TWG).

As the Cloud Storage industry evolves, it is also becoming increasingly clear that several cloud storage offerings are out there. These include:

• Elastic and extensible cloud storage for web-based media. App stores, music, eBooks, streaming videos and social media
• Cloud storage for Cloud-backup such as Mozy, Carbonite etc
• Cloud storage for file-based, collaboration and multi-device access. Dropbox and Dropbox-like solutions are the best examples
• Cloud storage for archive and records preservation
• Storage for cloud computing applications

Of course, the list above can only continue to grow.

And we are seeing the changing of how applications in the web access data from storage. It is no longer the file-based or block-based access, but more from these set of requirements:

• Access through a URL (Uniform Resource Locator) and an globally unique ID.
• Rich in metadata which allows interaction and feeding of data patterns into other application sets
• Self-contained, ready-to-use applications that natively links through HTTP/Restful/XML-RPC protocols and other web-based data format access methods and specifications (XML/SOAP/JSON/others).

Therefore, CDMI accesses cloud storage objects through the commonly used semantics of CRUD (Create, Retrieve, Update and Delete) as shown in the diagram below:

The diagram is from the slides of Mark Carlson of Oracle, who happens to be the other co-Chair of the SNIA Cloud Storage TWG.

I am still learning the CDMI specifications and the version 1.0.2 document is over 200 pages! As CDMI is basking on its status as an ISO standard, ISO/IEC 17826:2012, I as a storage networking guy has to adapt to incorporate cloud computing skills and knowledge into my repertoire.

And hopefully, having both the cloud and storage experience will make us storage networking guys even stranger and making us even more difficult to comprehend.

We are a strange lot. Indeed we are.