There is a mini revolution going on, and Facebook is the main force driving it.
It is the Open Compute Project (OCP), and its mission is to redesign the modern-day data centers and drive open hardware and architectural designs and specifications, including storage. The overall goals are to drive greater data center efficiency, flexibility, energy savings and cost effectiveness in a new class of “hyperscale” datacenters. Facebook, Google and Amazon are some of the examples of hyperscale datacenters, where their businesses relies on massive computing power, exponential storage performance and racks and racks of computing infrastructure to drive their web-computing or cloud-computing services.
Some of the cool technology innovations in mind includes having systems that support any CPUs from any vendors including Intel and AMD. We may even see both processor brands running on the same motherboard. The Open Common Slots component for processors is based on PCIe. Intel has pledged their Decathlete motherboard specifications for OCP and likewise AMD has produced its Roadrunner mobo series specification for the project as well. The ARM processor could also be supported in the near future in this “mix-and-match” OCP ideals.
Other proposed changes include OpenRack specifications, “sleds”, and of course, the Open Vault project for storage (aka “Knox”).
As the Open Vault Storage specification version 0.5 states, the hardware design of Open Vault is a 2U storage enclosure that holds up to 30 units of 3.5″ HDDs. Basically it is really 2 1U trays holding 15 HDDs each. Both 1U trays are interconnected via 4 SAS x4 6Gbps expander boards, 2 (SEB-A and SEB-B below) on each tray.
Here is a high level look at the system components layout of 1 tray:
Here is another view from the front, with both 1U trays together making it 2U.
There is also a photo I got from the specification document, which I like:
All these proposals to the OCP project are really fantastic because it looks like it is going to take off in a big, big way.
I have already seen these significant changes to storage space, as industry players, driven by server virtualization are “cutting the fat“ in between the compute layer and the storage layer, moving the I/O processing and communication closer and closer to the processing power.
From another point of view, pretty soon the storage networking industry would have to drop the “networking” portion and the storage landscape in the hyperscaling datacenters will be dearth of networking protocols such as iSCSI and Fibre Channel. It will be back to the days of DAS (direct-attached storage) once again, but this time, the networking will be based on channel-based, high performing, and low-latency communications such as PCIe.
The “addressing”, delivery methods and configuration will be handled by the software itself, running at the compute layer and we no longer have to configure IP for storage, iSCSI sessions or Fibre Channel zoning to link the LUNs to the applications.
And it is already happening today as we have seen how VMware VSA (virtual storage appliance) doing all that using NFS with internal direct-attached storage. Bigger things are coming too from VMware, like the Virtual SAN technology and it is going to disrupt the storage industry in a big way. I reiterate my stand that VMware is going to be a storage killer. Read my blog post here.
So, if Facebook and its Open Compute Project supporters have it their way, that could spell a paradigm shift for storage networking professionals (or should I say storage professionals). We would see a new breed of cloud computing professionals coming up in the near future that will have compute, networking, storage, applications, service delivery and business management skill sets, to dominate the job landscape.
At the same time, OCP could define how storage hardware is designed in the future for hyperscalability, creating a snowballing effect on the storage networking industry as a whole. The strongest effect, I believe, will be how future storage OSes are designed to fit into the Open Vault storage specifications.
Perhaps the least important requirement of the Open Vault specification is “vanity-free” but that could take away those ugly bezel designs I see nowadays on the storage hardware. I am hoping and praying … 🙂
The assimilation of compute and storage is seen as one of the early steps in the brave new world. As the Borgs would say, “Resistance is futile”.
This one Facebook likes …
Pingback: VMware in step 1 breaking big 6 hegemony | Storage Gaga
“Resistance is futile” – Still Captain Picard won in “The First Contact” 🙂
So, what do you then predict for the future of vendors in the space. Will it still be possible to innovate by providing a specialised product to assist people build this sort of infrastructure? Can people step up and gain proficiency in all of the skills you’ve named with enough numbers to service the market, of will there just be 4 or 5 massive cloud providers that provide services to the world?
What’s your vision for the future?
Hi Paul
Great to hear from you! Very sorry for the late reply. There was much travelling and also the SNIA IIIS event in Malaysia and Singapore last week.
We are already seeing IT services being moved to the cloud and companies like Facebook, Google, AWS are already building hyperscaling data centers to meet this demand. And the secret sauce for storage vendors to maintain is software.
History repeats itself when we have seen the mainframe moving to a niche solution, the Balkanization from the Unix wars in the 90s led the rise of the x86 platform with Windows NT and later, and also with Linux. We are also seeing the maturity of the server virtualization market and VMware is struggling to reinvent itself to become a Cloud Service Provider. All these happenings are leading towards a white landscape, where storage vendors’ brands become irrelevant.
The growth of the data also mean that the physics in storage hardware is not keeping up as well. We have seen massive rise in spinning drives capacity, but the performance per spindle remained the same. Relatively, I/O performance in fact, has decreased. The solid states are addressing the performance need but not the capacity need (yet). The massive data landscape also means that designs of data placement have to change as well. AWS S3, OpenStack Swift, are pushing for replication of data rather than RAID and so on.
All these are leading to the federation of abstraction of infrastructure and the rise of services.
IT skills have to adapt to many areas as well. Compute, network, storage, desktops, applications, database, DevOps, security and many more. One has to be multi-faceted, Jack-of-all-trades, master-of-all as well. That’s a challenge because there are not enough skillset and experience to meet this demand. I read an article interviewing Paul Gelsinger, VMware’s CEO. Only 5% meet that skill requirement bracket right now.
That’s my view of things.
I wish you well in your new position at Commvault.
Thank you
/Chin-Fah