Is General Purpose Object Storage disenfranchised?

[Disclosure: I am invited by GestaltIT as a delegate to their Storage Field Day 19 event from Jan 22-24, 2020 in the Silicon Valley USA. My expenses, travel, accommodation and conference fees will be covered by GestaltIT, the organizer and I am not obligated to blog or promote the vendors’ technologies to be presented at this event. The content of this blog is of my own opinions and views]

This is NOT an advertisement for coloured balls.

This is the license to brag for the vendors in the next 2 weeks or so, as we approach the 2020 new year. This, of course, is the latest 2019 IDC Marketscape for Object-based Storage, released last week.

My object storage mentions

I have written extensively about Object Storage since 2011. With different angles and perspectives, here are some of them:

• The Future is Intelligent Objects (2011)
• What should be Cloud Storage? (2011)
• APIs that stick in Storage (2012)
• Has Object Storage become the Everything Store? (2013)
• Of Object Storage, Filesystems and Multicloud (2017)
• My Dilemma of Stateful Storage Marriage (2018)
• The Malaysian Openstack Storage Conundrum (2018)
• Sleepless in Malaysia with Object Storage (2019)
• The Waning Light of Openstack Swift (2019)

The race to the bottom

Thus far since last year, the conversations of General Purpose Object Storage have begun to wet the lips of the Malaysian IT crowd. This is a good thing but technical awareness is still shallow, and GP Object Storage is almost always associated to a price per TB. This awareness has been strengthen over the years by the rise and the popularity (and extremely cheap) of S3 (Simple Storage Service), so much so that S3 is the epitome of Object Storage.

In the past 2 years of my observations, many Malaysian hosting companies and larger enterprises have tried to replicate the S3 service, with a varying degree of success (more about this in the next paragraph), and with a high degree of implementation complexity. With a mix of open source implementation such as RedHat Ceph , niche vendors like Cloudian and proprietary tech in the form of Hitachi Content Platform and NetApp StorageGrid. Somewhere in between, I know of IBM Malaysia desperately trying to reinvigorate their Cleversafe investment, now rebranded as iCOS (Cloud Object Storage). And I have not mentioned the tepid presence of Dell EMC ECS Object Storage in Malaysia. And I was also approached by Scality years ago to reach out to Petronas, Malaysia’s National Oil and Gas company, given my network with the Oil & Gas upstream community here.

The degree of success, as I have known (unless I am very wrong) has been mixed because the lowest price per TB has been the determinant of success. Those who chose open sourced object storage like Ceph were trying to maximize their margins with the lowest price possible, whilst enterprises using object storage perceived them as the lowest tier of disk-based storage. Those who have weaned from tapes saw the object storage target as a backup and archive repository tier, and therefore worthy only if the enterprises’ investment price of the platform was low to begin with.

This created a deeply ingrained set of perceptions that object storage has to be

• Slow – not suitable for transactional workloads
• Cheap – lowest cost per TB
• Reliable – Expecting 13x9s but enterprises do not even invest in a secondary replication target (refer to previous bullet point #2)
• Cloud-like – Many still want it to be like Dropbox, Google Drive or Microsoft OneDrive. (that mismatched idea that object storage should be a file share and synced with all the iOS, Android devices)

These associated perceptions of Object Storage has started the self disenfranchising path years ago. This blog intends to change those perceptions and return the dignity of Object Storage, at least for the Malaysian IT crowd.

The General Purpose Object Storage Kerfuffle

I wrote about OpenStack Swift a few months ago. This obviously ruffled some feathers (pun intended) with a couple of SwiftStackers, who reached out to clear the fuzziness I probably created. That was in August this year.

Swiftstack has been in the General Purpose Object Market for some years now. They are in the IDC Marketscape chart above. When the news broke out last week that they had to let go of some of their staff in the US, and pivoting to a newer focus of higher, more transactional workloads, this led to many thoughts in my mind.

Obviously, this to me, is a signal that the General Purpose Object Storage is looking like a kerfuffle of racing to the bottom, one which may not be lucrative in the coming years. Swiftstack must have the visionary goggles to see this happening soon, and thus refocusing their efforts into areas such as “deep learning (autonomous vehicle development), personalized medicine (genomics), telco/5G (edge computing), and analytics (esports, logistics)”.

The smartest one is Minio

Missing from the latest IDC chart is MinIO. When this chart came out, several notable folks commented that MinIO was missing, probably with a tinge of disappointment. I was elated that MinIO was not listed because they are on a totally different blue ocean strategy. Look at this non-analyst chart.

The purple square is exactly the IDC Marketscape for Object-based Storage. General-purpose, slow, cheap, resilient (maybe) and cloud (wrong perception dudes!). And MinIO is in a different class, a different category and designed from ground up to be a high performant platform for highly transactional workloads and extending it easily to the cloud. The Big Red Dot is where MinIO excels.

Many high performance benchmark reports were already done by MinIO, and the simplicity to craft an Object Storage with MinIO for modern workloads such as deep learning, machine learning and cloud native high performance computing is just what it is. Simple!

One of the compelling reports is the recent Gartner Peer Insights, and I encourage all to check it out.

Workloads for data is changing

Workloads for data is changing under our feet as we speak. The Hadoop implosion was the perfect example.The market landscape has changed swiftly and the General Purpose Object Storage market is facing a clear and present danger if we continue to apply the same notion and perspective of what it is now.

What is next?

Throwing flash storage to an archaic object storage architecture is probably not a great idea. NetApp did that in the recent Insight show. That was just retrofitting. I am sure many would follow suit, and it would not be convincing at all.

I read this quote when I was a Computer Science student in the US many aeons ago. Presumably from the ice hockey great Wayne Gretzky,

it is time to think different about object storage.

What is next is where I hope to learn again from AB, CEO and Founder of MinIO. I am looking for our meeting at Storage Field Day 19 next month. I can’t wait.

Full disclosure: My company, Katana Logic, has been a MinIO reseller in Malaysia for almost 2 years now.