Data services platforms – 2018 and beyond

2017 is drawing to a close. Sadly, I was greeted with the news of Oracle laying off their storage hardware sales team yesterday. And I couldn’t help to see where all this is going, the Oracle Cloud. The cloud has become the data services platform of choice, not on-premise storage infrastructure anymore.

Years ago, when I first started blogging, I wrote that Cloud Computing could make you lose your job unless… I don’t usually make predictions but that prediction in 2011 is becoming true and more prevalent.

What is the future looking like?

For one, it is not bleak and plenty to look forward to. Forrester predicts that in 2018 (or at the end of it), Amazon AWS, Microsoft Azure and Google will capture 76% of all cloud platform revenue and 80% by 2020. More data will be generated at the edge than it is being created centrally on public clouds. The demand for high performance data services platforms will be beyond your usual object-based storage and having a data singularity where data can transcend across premises will become crucial in maintaining, extending and improving the services from core-to-edge. Multi-Cloud or Cross-Cloud services platforms are maturing because the cloud platform space, while dominated by AWS, Azure and Google, includes IBM Cloud, Oracle Cloud, Rackspace, Alibaba Cloud, and is also about localized and regionalized players like Markely, Virtustream and ReScale serving unique and niche markets.

To address this shift, data services platforms are reinventing itself to be different. Flash-based, NVMe storage (err, I mean data services platform) is the foundation of building and drive self-service analytics, whether it is file-aware or content-aware, or infrastructure-aware. This new found “awareness” would inculcate platform intelligence and data intelligence, driving automation towards predictive and preemptive actions.

From a security point of view, data privacy and data governance take precedence of form and shape. As Europe enforces the General Data Protection Regulation in May of 2018, the proliferation of multi-clouds and cross-clouds will be questions. How safe is my organization’s data? How will it be regulated as the data crosses cloud boundaries? How to ensure that data workflows and pipelines move freely to shared and unencumbered? These questions are surely be eyeballed in any data regulated segments of the businesses and the individuals who have dealings in those markets.

What about people like us who have been in the storage technology industry for a lot time? I have reverberated that a technology person doing technical work has stand out. Going back to my old 2011 blog, you have to be better than better, knowing the technologies deeper than deeper, and be more connected than you are connected right now. Be EXTRAORDINARY than the typical run-of-the-mill engineer or consultant or architect. Stand Out!

This is not a prediction for the future. I am not a futurist but the signs of change upon the data services platforms (storage for you dinosaurs, yours truly included) are shaping up to tangible forms. And we are going to see lots of more disruptive stuff in 2018 and beyond.

Just my once-in-a-while ranting and we will have a fantastic 2018!

 

Of Object Storage, Filesystems and Multi-Cloud

Data storage silos everywhere. The early clarion call was to eliminate IT data storage silos by moving to the cloud. Fast forward to the present. Data storage silos are still everywhere, but this time, they are in the clouds. I blogged about this.

Object Storage was all the rage when it first started. AWS, with its S3 (Simple Storage Service) offering, started the cloud storage frenzy. Highly available, globally distributed, simple to access, and fitted superbly into the entire AWS ecosystem. Quickly, a smorgasbord of S3-compatible, S3-like object-based storage emerged. OpenStack Swift, HDS HCP, EMC Atmos, Cleversafe (which became IBM SpectrumScale), Inktank Ceph (which became RedHat Ceph), Bycast (acquired by NetApp to be StorageGrid), Quantum Lattus, Amplidata, and many more. For a period of a few years prior, it looked to me that the popularity of object storage with an S3 compatible front has overtaken distributed file systems.

What’s not to like? Object storage are distributed, they are metadata rich (at a certain structural level), they are immutable (hence secure from a certain point of view), and some even claim self-healing (depending on data protection policies). But one thing that object storage rarely touted dominance was high performance I/O. There were some cases, but they were either fronted by a file system (eg. NFSv4.1 with pNFS extensions), or using some host-based, SAN-client agent (eg. StorNext or Intel Lustre). Object-based storage, in its native form, has not been positioned as high performance I/O storage.

A few weeks ago, I read an article from Storage Soup, Dave Raffo. When I read it, it felt oxymoronic. SwiftStack was just nominated as a visionary in the Gartner Magic Quadrant for Distributed File Systems and Object Storage. But according to Dave’s article, Swiftstack did not want to be “associated” with object storage that much, even though Swiftstack’s technology underpinning was all object storage. Strange.

Continue reading

DellEMC SC progressing well

[Preamble: I was a delegate of Storage Field Day 14. My expenses, travel and accommodation were paid for by GestaltIT, the organizer and I was not obligated to blog or promote the technologies presented at this event. The content of this blog is of my own opinions and views]

I haven’t had a preview of the Compellent technology for a long time. My buddies at Impact Business Solutions were the first to introduce the Compellent technology called Data Progression to the local Malaysian market and I was invited to a preview back then. Around the same time, I also recalled another rather similar preview invitation by PTC Singapore for the 3PAR technology called Adaptive Provisioning (it is called Adaptive Optimization now).

Storage tiering was on the rise in the 2009-2010 years. Both Compellent and 3PAR were neck and neck leading the conversation and mind share of storage tiering, and IBM easyTIER and EMC FAST (Fully Automated Storage Tiering) were nowhere to be seen or heard. Vividly, the Compellent Data Progression technology was much more elegant compared to the 3PAR technology. While both intelligent storage tiering technologies were equally good, I took that the 3PAR founders were ex-Sun Microsystems folks, and Unix folks sucked at UX. In this case, Compellent’s Data Progression was a definitely a leg up better than 3PAR.

History aside, this week I have the chance to get a new preview of the Compellent technology again. Compellent was now rebranded as the SC series and was positioned as the mid-range storage arrays of DellEMC. And together with the other Storage Field Day 14 delegates, I have the pleasure to experience the latest SC Data Progression technology update, as well their latest SC All-Flash.

In Data Progression, one interesting feature which caught my attention was the RAID Tiering. This was a dynamic auto expand and auto contract set of RAID tiersRAID 10 and RAID 5/6 in the Fast Tier and RAID 5/6 in the Lower Tier. RAID 10, RAID 5 and RAID 6 on the same set of drives (including SSDs), and depending on the “hotness” of the data, the location of the data blocks switched between the several RAID tiers in the Fast Tier. Over a longer period, the data blocks would relocate transparently to the Capacity Tier from the Fast Tier.

The Data Progression technology is extremely efficient. The movement of the data between the RAID Tiers and between the Performance/Capacity Tiers are in pages instead of blocks, making the write penalty and bandwidth to a negligible minimum.

The Storage Field Day 14 delegates were also privileged to be the first to get into the deep dive of the new All-Flash SC, just days of the announcement of the All-Flash SC. The All Flash SC redefines and refines the Data Progression to the next level. Among the new optimization, NAND Flash in the SC (both SLCs and MLCs, read-intensive and write-intensive) set the Data Progression default page size from 2MB to 512KB. These smaller 512KB pages enabled reduced bandwidth for tiering between the write-intensive and the read-intensive tier.

I didn’t get the latest SC family photos yet, but I managed to grab a screenshot of the announcement from The Register of the new DellEMC SC Series.

I was very encouraged with the DellEMC Midrange Storage presentation. Besides giving us a fantastic deep dive about the DellEMC SC All-Flash Storage, I was also very impressed by the candid and straightforward attitude of the team, led by their VP of Product Management, Pierluca Chiodelli. An EMC veteran, he was taking up the hard questions onslaught by the SFD14 delegate like a pro. His team’s demeanour was critical in instilling confidence and trust in how the bloggers and the analysts viewed Dell EMC merger, and how the SC and the Unity series would pan out in the technology roadmap.

Unlike the fiasco I went through with the DellEMC Forum 2017 in Malaysia, where I was disturbed with 3 calls in 3 consecutive days by DellEMC Malaysia, I was left with a profound respect for this DellEMC Storage team. They strongly supported their position within the DellEMC storage universe, and imparted their confidence in their technology solution in the marketplace.

Without a doubt, in my point of view, this DellEMC Mid-Range Storage team was the best I have enjoyed in Storage Field Day 14. Thank you.

Commvault UDI – a new CPUU

[Preamble: I am a delegate of Storage Field Day 14. My expenses, travel and accommodation are paid for by GestaltIT, the organizer and I am not obligated to blog or promote the technologies presented at this event. The content of this blog is of my own opinions and views]

I am here at the Commvault GO 2017. Bob Hammer, Commvault’s CEO is on stage right now. He shares his wisdom and the message is clear. IT to DT. IT to DT? Yes, Information Technology to Data Technology. It is all about the DATA.

The data landscape has changed. The cloud has changed everything. And data is everywhere. This omnipresence of data presents new complexity and new challenges. It is great to get Commvault acknowledging and accepting this change and the challenges that come along with it, and introducing their HyperScale technology and their secret sauce – Universal Dynamic Index.

Continue reading

Commvault calling again

[Preamble: I will be a delegate of Storage Field Day 14. My expenses, travel and accommodation are paid for by GestaltIT, the organizer and I am not obligated to blog or promote the technologies presented in this event]

I am off to the US again next Monday. I am attending Storage Field Day 14 and it will be a 20+ hour long haul flight. But this SFD has a special twist, because I will be Washington DC first for Commvault GO 2017 conference. And I can’t wait.

My first encounter with Commvault goes way back in early 2001. I recalled they had their Galaxy version but in terms of market share, they were relatively small compared to Veritas and IBM at the time. I was with NetApp back then, and customers in Malaysia hardly heard of them, except for the people in Shell IT International (SITI). For those of us in the industry, we all knew that SITI worldwide had an exclusive Commvault fork just for them.

Continue reading

Cloud silos after eliminating silos

I love cloud computing. I love the economics and the agility of the cloud and how it changed IT forever. The cloud has solved some of the headaches of IT, notably the silos in operations, the silos in development and the silos in infrastructure.

The virtualization and abstraction of rigid infrastructures and on-premise operations have given birth to X-as-a-Service and Cloud Services. Along with this, comes cloud orchestration, cloud automation, policies, DevOps and plenty more. IT responds well to this and thus, public clouds services like Amazon Web Services, Microsoft Azure, and Google Cloud Platforms are dominating the landscape. Other cloud vendors like Rackspace, SoftLayer, Alibaba Cloud are following the leaders pack offering public, private, hybrid and specialized services as well.

In this pile, we can now see the certain “camps” emerging. Many love Azure Stack and many adore AWS Lambda. Google just had their summit here in Malaysia yesterday, appealing to a green field and looking for new adopters. What we are seeing is we have customers and end users adopting various public cloud services providers, their services, their ecosystem, their tools, their libraries and so on. We also know that many customers and end users having several applications on AWS, and some on Azure and perhaps looking for better deals with another cloud vendor. Multi-cloud is becoming flavour of the season, and that word keeps appearing in presentations and conversations.

Yes, multi-cloud is a good thing. Customers and end users would love it because they can get the most bang for their buck, if only … it wasn’t so complicated. There aren’t many “multi-cloud” platforms out there yet. Continue reading

DellEMC Forum 2017 – A contrast of worlds

The DellEMC Forum 2017 in Kuala Lumpur concluded yesterday. I was there to catch up with old friends, pick up gossips in the grapevine but most of all, to really see how the new DellEMC was doing.

From the news and sources for the past 1 year, everything looked fine and dandy. In fact, I was super impressed in the way the whole merger thing was going, because DellEMC was firing at all fronts.

And the DellEMC Forum started with some good energy. They brought out many of the storage, hyper-converged, software-defined solutions. On show, they had XtremIO, Scale IO, Isilon, and even a full hands-on lab booth at the corner end of the floor. DellEMC was also inclusive the SMB/SME segment, dedicating a separate pavilion for the SMB/SME solutions.

Continue reading

Pure Electric!

I didn’t get a chance to attend Pure Accelerate event last month. From the blogs and tweets of my friends, Pure Accelerate was an awesome event. When I got the email invitation for the localized Pure Live! event in Kuala Lumpur, I told myself that I have to attend the event.

The event was yesterday, and I was not disappointed. Coming off a strong fiscal Q1 2018, it has appeared that Pure Storage has gotten many things together, chugging full steam at all fronts.

When Pure Storage first come out, I was one of the early bloggers who took a fancy of them. My 2011 blog mentioned the storage luminaries in their team. Since then, they have come a long way. And it was apt that on the same morning yesterday, the latest Gartner Magic Quadrant for Solid State Arrays 2017 was released.

Continue reading

The rise of RDMA

I have known of RDMA (Remote Direct Memory Access) for quite some time, but never in depth. But since my contract work ended last week, and I have some time off to do some personal development, I decided to look deeper into RDMA. Why RDMA?

In the past 1 year or so, RDMA has been appearing in my radar very frequently, and rightly so. The speedy development and adoption of NVMe (Non-Volatile Memory Express) have pushed All Flash Arrays into the next level. This pushes the I/O and the throughput performance bottlenecks away from the NVMe storage medium into the legacy world of SCSI.

Most network storage interfaces and protocols like SAS, SATA, iSCSI, Fibre Channel today still carry SCSI loads and would have to translate between NVMe and SCSI. NVMe-to-SCSI bridges have to be present to facilitate the translation.

In the slide below, shared at the Flash Memory Summit, there were numerous red boxes which laid out the SCSI connections and interfaces where SCSI-to-NVMe translation (and vice versa) would be required.

Continue reading