Is Software Defined right for Storage?

George Herbert Leigh Mallory, mountaineer extraordinaire, was once asked “Why did you want to climb Mount Everest?“, in which he replied “Because it’s there“. That retort demonstrated the indomitable human spirit and probably exemplified best the relationship between the human being’s desire to conquer the physical limits of nature. The software of humanity versus the hardware of the planet Earth.

Juxtaposing, similarities can be said between software and hardware in computer systems, in storage technology per se. In it, there are a few schools of thoughts when it comes to delivering storage services with the notable ones being the storage appliance model and the software-defined storage model.

There are arguments, of course. Some are genuinely partisan but many a times, these arguments come in the form of the flavour of the moment. I have experienced in my past companies touting the storage appliance model very strongly in the beginning, and only to be switching to a “software company” chorus years after that. That was what I meant about the “flavour of the moment”.

Software Defined Storage

Software is winning but the storage appliance?

Last week, Vast Data announced their Gemini Storage Consumption business model. They are exiting their previous storage appliance-like business to become a software-only business, ala Nutanix®. Nutanix® exited the hardware business several years ago, but not without early struggles. So, the questions are, will a storage appliance business model thrive and will a software defined storage model the right move for the future? We explore several perspectives in this blog.

Open source opens up software defined storage

In my views, the open source software model really opened the software defined storage market segment two decades ago. The defining moment for me was when Oracle® released their 9i RAC (Real Application Cluster) on Linux in 2001. At the time, I was already spending my weekends learning Red Hat® Linux after switching my Sun® Solaris allegiance and preparing for my Red Hat® RHCE® certification. Oracle® 9i RAC was perfect for me because I could get Oracle® CDs easily via Oracle® Technology Network (OTN). I spent a lot of weekends honing my Linux and Oracle® skills during those many moons.

The open source movement nudged those enterprise storage vendors’ frozen proprietary permafrost ideologies, most of them touting the storage appliance model at the time, towards software-defined, and past storage philosophies were thawing. The Intel® x86 architecture (to a smaller extent, a burgeoning x86-64/AMD64 architecture) fueled the open source revolution as well, with enterprise-grade software became easily and readily available in the hands of the masses, people on the streets like me. NetApp®, whom I was working for at the time, began to shift and market themselves as a software company. Many enterprise storage companies at the time, pure play storage vendors, were also doing the same song and dance. “We are a software company”.

By the end of that decade, circa 2010-2012, enterprise-grade software defined storage companies either proprietary or open sourced like Nexenta® (acquired by DDN) Scality, Cleversafe (acquired by IBM®), Gluster (acquired by Red Hat®) , Coraid were aplenty.

With my roots in Sun® Solaris and Red Hat® Linux, I naturally took up Nexenta® CORE and NexentaStor (my first business, Next Logic became a Nexenta® reseller in 2009), Gluster and Coraid. I wrote a few blogs about the software defined storage industry at the time:

[ Coraid ] All about Ethernet
[ Gluster ] Red Hat to acquire Gluster
[ Illumos ] Phoenix rising from OpenSolaris ashes
[ General ] “I want to put in my own hard disks”

Going the other way?

Not surprisingly, despite the flourishing software-defined storage market, there are still many storage and hyperconverged vendors that have not gone 100% software only, despite their software-defined marketing. Storage vendors like NetApp®, Hitachi Vantara and Pure Storage® are still plying their trade with their storage appliances. Several startups are straddled between the full storage appliance and the software-defined storage models. A few come to mind, past and present, which often has a piece of offloading hardware component, such as:

Thus, there are merits when a storage hardware technology is involved, especially when it comes to specialized workloads and accelerated performance. The best examples are the high performance and supercomputing industry segments. Hardware technology has its place in storage.

Moreover, there is a mini storage technology tide rising in the form of DPUs (data processing units) and Computational Storage, which is bucking the trend to go 100% software-defined storage. These will be future topics for another day.

Purpose-built romance

There is romanticism when it comes to a well-built appliance. Both software and hardware working flawlessly and in harmony. Apple® iPhone and Mac users can certainly attest to that experience, because the purpose-built appliance is a thing of beauty. That was also one of the reasons why NetApp® was able to rise rapidly in the world of enterprise storage through the years of 2000s. For the millennials, NetApp® used to be called Network Appliance®.

Old NetApp logo – previously known as Network Appliance

IMHO, the symbiosis of the storage appliance, translating to ease-of-use, simplicity and highly optimized storage prowess, presents an omnipotent experience that cannot be described in mere marketing words.

Conclusion

The software-defined storage model has been and is wildly successful. IMHO, without a software driven storage technology, the storage hardware will not be as feature rich because there are so many tacts that the storage software technology can define and solve. Hardware-only storage, for simplicity a JBOD (just a bunch of disks) for example, well, will be just a bunch of disks. It is the storage software that makes the storage hardware turkey (pardon the expression) shine like a supernova.

On the flip side, storage hardware can make the storage software look like a million bucks. Without the specialized hardware components such as FPGAs (field programmable gate arrays), low latency interfaces and interconnects, offloading storage processors and other bleeding edge hardware, the storage software developed for the commodity hardware can only go so far. You will never be a Lewis Hamilton if the car that you drive is a Trabant .

Imagine Lewis Hamilton driving a Trabant in an F1 race

To conclude, the software defined storage model is right for the industry. But both the software and the hardware technologies developments have commoditized the industry as well as pushing new boundaries to store, process and share data in ways that are faster, simpler, better, cheaper. This has been the ebb and flow of how the storage technology industry has changed and moved forward, with hardware and software breaking new grounds and innovate.

My parting shot to those storage vendors who want to call their storage solution “storageless“. It is hypocritical to have a software-defined storage technology and yet still use some storage hardware to complete the solution, even if the storage hardware is commoditized. As the saying goes, you can’t clap without the other hand.