Elastifile Archives - Storage Gaga

Trusting your storage – It’s not about performance

By cfheoh | December 28, 2020 - 9:00 am |December 26, 2020 Acquisition, Appliance, Backup, Business Continuity, Cloud, Data, Data Corruption, Data Protection, Elastifile, EMC, Filesystems, Flash, NetApp, OpenZFS, RAID

6 Comments

I have taken some downtime from my blog since late October. Part of my “hiatus” was my illness which had affected my right kidney but I am happy to announce that I am well again. During this period, I spent a lot of the time reading the loads of storage technologies announcements and their marketing calls and almost every single one of them touts Performance as if it is the single “sellable” feature of the respective storage vendor. None ever positions data integrity and the technology behind it in what I believe as the most important and most fundamental feature of any storage technology – Reading the right data exactly it was written into the storage array.

[ Note: Data integrity is even more critical in cloud storage and data corruption, especially the silent ones are even more acute in the clouds ]

Sure, this fundamental feature sounds like it is a given thing in any storage array but believe me, there are enterprise storage arrays which have failed to deliver this simple feature properly. I have end users coming to me through out my storage career that they have database corruption, or file corruption and unable to access their data in an acceptable manner. Data corruption is real folks!

Data corruption.

After several weeks of reading these stuff, I got jaded with so many storage vendors playing leapfrog announcements with their millions of IOPS boasts.

The 3 legged stool

Rewind to circa 2012, just about the time when EMC® acquired XtremIO™. XtremIO™ was a nascent All-Flash startup, and many, including yours truly, really saw the EMC® acquisition was about a high performant storage array. I was having an email conversation with Shahar Frank, one of the co-founders of XtremIO™, and expressing my views about their performance. What Shahar replied surprised me.

The fundamentals of the strength of a storage array was a like a 3-legged stool. 2 legs of the stool would be Performance, and Protection, but with 2 legs, the person sitting on the stool would fall. The 3rd leg would stabilize the balance of the stool, and this 3rd leg was Reliability. This stumped me because XtremIO™’s most sellable feature was Performance. But the wisdom of Shahar pointed to Reliability, the least exciting feature and the most dull of the 3. He was brilliant, of course and went on to found ElastiFile (acquired by Google™), but that’s another story for another day.

Continue reading →

Down the rabbit hole with Kubernetes Storage

By cfheoh | May 19, 2020 - 9:30 am |May 16, 2020 Acquisition, Algorithm, Amazon Web Services, Analytics, API, Artificial Intelligence, Ceph, Cloud, Clusters, Containers, Data Management, Edge Computing, Elastifile, Filesystems, Flash, Google, Hyperconvergence, Kubernetes, Linux, Minio, NFS, Object Storage

Storage Performance Considerations for AI Data Paths

By cfheoh | June 17, 2019 - 10:50 am |June 17, 2019 100Gigabit Ethernet, Algorithm, Analytics, API, Artificial Intelligence, Big Data, Cloud, Composable Infrastructure, Data, Data Fabric, Data Management, Data Privacy, Data Security, Digital Transformation, Drivescale, E8 Storage, Edge Computing, Elastifile, Excelero, Filesystems, High Performance Computing, Hyperconvergence, Industry 4.0, Infiniband, Intel, Liqid, Lustre, Machine Learning, Mellanox Technologies, NVMe, Object Storage, Performance Benchmark, Performance Caching, Quantum Corporation, RDMA, Software-defined Datacenter, Storage Optimization, Storage Tiering, ThinkParq, Vast Data, Virtualization, WekaIO

1 Comment

The hype of Deep Learning (DL), Machine Learning (ML) and Artificial Intelligence (AI) has reached an unprecedented frenzy. Every infrastructure vendor from servers, to networking, to storage has a word to say or play about DL/ML/AI. This prompted me to explore this hyped ecosystem from a storage perspective, notably from a storage performance requirement point-of-view.

One question on my mind

There are plenty of questions on my mind. One stood out and that is related to storage performance requirements.

Reading and learning from one storage technology vendor to another, the context of everyone’s play against their competitors seems to be “They are archaic, they are legacy. Our architecture is built from ground up, modern, NVMe-enabled“. And there are more juxtaposing, but you get the picture – “We are better, no doubt“.

Are the data patterns and behaviours of AI different? How do they affect the storage design as the data moves through the workflow, the data paths and the lifecycle of the AI ecosystem?

Continue reading →

Figuring out storage for Kubernetes and containers

By cfheoh | April 16, 2019 - 6:44 pm |April 16, 2019 Amazon Web Services, API, Cloud, Containers, Data, Data Archiving, Data Availability, Data Management, Data Protection, Data Security, Docker, Edge Computing, Elastifile, Fibre Channel, Filesystems, Google, Hyperconvergence, Kubernetes, NFS, Openstack, Performance Benchmark, Performance Caching, Robin.io, Snapshots, Software Defined Storage, Storage Optimization, Virtualization, VMware

2 Comments

Oops! I forgot about you!

To me, containers and container orchestration (CO) engines such as Kubernetes, Mesos, Docker Swarm are fantastic. They scale effortlessly and are truly designed for cloud native applications (CNA).

But one thing irks me. Storage management for containers and COs. It was as if when they designed and constructed containers and the containers orchestration (CO) engines, they forgot about the considerations of storage and storage management. At least the persistent part of storage.

Over a year ago, I was in two minds about persistent storage, especially when it comes to the transient nature of microservices which was so prevalent and were inundating the cloud native applications landscape. I was searching for answers in my blog. The decentralization of microservices in containers means mass deployment at the edge, but to have the pre-processed and post-processed data stick to the persistent storage at the edge device is a challenge. The operative word here is “STICK”.

Two different worlds

Containers were initially designed and built for lightweight applications such as microservices. The runtime, libraries, configuration files and dependencies are all in one package. They were meant to do simple tasks quickly and scales to thousands easily. They could be brought up and brought down in little time and did not have to bother about the persistent data stored by the host. The state of the containers were also not important to the application tasks at hand.

Today containers like Docker have matured to run enterprise applications and the state of the container is important. The applications must know the state and the health of the container. The container could be in online mode, online but not accepting data mode, suspended mode, paused mode, interrupted mode, quiesced mode or halted mode. Each mode or state of the container is important to the running applications and the container can easily brought up or down in an instance of a command. The stateful nature of the containers and applications is critical for the business. The same situation applies to container orchestration engines such as Kubernetes.

Container and Kubernetes Storage

Docker provides 3 methods to local storage. In the diagram below, it describes:

Continue reading →

Storage dinosaurs evolving too

By cfheoh | March 7, 2018 - 8:50 pm |March 8, 2018 Analytics, API, Appliance, CIFS, Cloud, Data, Data Fabric, Data Management, Elastifile, FCoE, Fibre Channel, Hedvig, High Performance Computing, Hyperconvergence, Infiniband, iSCSI, NAS, NVMe, Object Storage, Openstack, RDMA, Scale-out architecture, SNIA, Storage Field Day, Uncategorized, Virtualization, WekaIO

2 Comments

[Preamble: I am a delegate of Storage Field Day 15 from Mar 7-9, 2018. My expenses, travel and accommodation are paid for by GestaltIT, the organizer and I am not obligated to blog or promote the technologies presented at this event. The content of this blog is of my own opinions and views]

I have been called a dinosaur. We storage networking professionals and storage technologists have been called dinosaurs. It wasn’t offensive or anything like that and I knew it was coming because the writing was on the wall, … or is it?

The cloud and the breakneck pace of all the technologies that came along have made us, the storage networking professionals, look like relics. The storage guys have been pigeonholed into a sunset segment of the IT industry. SAN and NAS, according to the non-practitioners, were no longer relevant. And cloud has clout (pun intended) us out of the park.

I don’t see us that way. I see that the Storage Dinosaurs are evolving as well, and our storage foundational knowledge and experience are more relevant that ever. And the greatest assets that we, the storage networking professionals, have is our deep understanding of data.

A little over a year ago, I changed the term Storage in my universe to Data Services Platform, and here was the blog I wrote. I blogged again just before the year 2018 began.

Continue reading →

My dilemma of stateful storage marriage

By cfheoh | February 17, 2018 - 1:28 pm |February 17, 2018 API, Cloud, Data Management, Datera, Elastifile, Filesystems, Hedvig, High Performance Computing, Hyperconvergence, NFS, Object Storage, OpenIO, Scality, Storage Field Day, Uncategorized, Virtualization, VMware, WekaIO

2 Comments

I should be a love match maker.

I have been spending much hours in the past few months, thinking of stateful data in stateful storage containers and how they would consummate with distributed applications containers and functions-as-a-service (aka serverless, aka Lambda). It still hasn’t made much sense, and I have not solved this problem yet. Although there were bits and pieces that coming together and the jigsaw looked well enough to give a cackled reply, what I have now is still not good enough for me. I am still searching for answers, better than the ones I have now.

The CAP theorem is in center of my mind. Distributed data, distributed states of data are on my mind. And by the looks of things, the computing world is heading towards containers and serverless computing too. Both distributed applications containers and serverless computing make a lot of sense. If we were to engage a whole new world of fog computing, edge computing, IoT, autonomous systems, AI, and other real-time computing, I would say that the future belongs to decentralization. Cloud Computing and having edge systems and devices getting back to the cloud for data is too slow. The latency of micro- or even nano-seconds is just not good enough. If we rely on the present methods to access the most relevant data, we are too late.

Continue reading →

Always serendipitous Storage Field Days

By cfheoh | February 13, 2018 - 12:15 pm |February 13, 2018 Cloud, Data Management, Elastifile, Hedvig, Storage Field Day, WekaIO

2 Comments

I am off to the Silicon Valley again in the first week of March. Storage Field Day 15 happens from March 7-9, 2018 and this will be my 4th time as a delegate. I am happy because there is always something new, something exciting, and on every occasion, serendipity happened for me.

There is plenty on my mind for this trip. There are several exciting technology vendors I am keen to learn from, notably Hedvig and WekaIO. With so much talks about multicloud and cloud lock-ins these days, there are a bunch of questions on my mind, and a few more that I am trying to piece together.

Continue reading →

Of Object Storage, Filesystems and Multi-Cloud

By cfheoh | November 22, 2017 - 12:25 pm |November 22, 2017 Amazon, CIFS, Cloud, Cloudian, Data Availability, Data Fabric, Data Management, Elastifile, Filesystems, High Performance Computing, Hyperconvergence, Nasuni, NFS, Object Storage, OpenIO, Openstack, Performance Benchmark, Performance Caching, Reliability, Scale-out architecture, Scality, Server SAN, SMB, Software Defined Storage, Software-defined Datacenter, Storage Optimization, swiftstack, Uncategorized, Virtualization

1 Comment

Data storage silos everywhere. The early clarion call was to eliminate IT data storage silos by moving to the cloud. Fast forward to the present. Data storage silos are still everywhere, but this time, they are in the clouds. I blogged about this.

Object Storage was all the rage when it first started. AWS, with its S3 (Simple Storage Service) offering, started the cloud storage frenzy. Highly available, globally distributed, simple to access, and fitted superbly into the entire AWS ecosystem. Quickly, a smorgasbord of S3-compatible, S3-like object-based storage emerged. OpenStack Swift, HDS HCP, EMC Atmos, Cleversafe (which became IBM SpectrumScale), Inktank Ceph (which became RedHat Ceph), Bycast (acquired by NetApp to be StorageGrid), Quantum Lattus, Amplidata, and many more. For a period of a few years prior, it looked to me that the popularity of object storage with an S3 compatible front has overtaken distributed file systems.

What’s not to like? Object storage are distributed, they are metadata rich (at a certain structural level), they are immutable (hence secure from a certain point of view), and some even claim self-healing (depending on data protection policies). But one thing that object storage rarely touted dominance was high performance I/O. There were some cases, but they were either fronted by a file system (eg. NFSv4.1 with pNFS extensions), or using some host-based, SAN-client agent (eg. StorNext or Intel Lustre). Object-based storage, in its native form, has not been positioned as high performance I/O storage.

A few weeks ago, I read an article from Storage Soup, Dave Raffo. When I read it, it felt oxymoronic. SwiftStack was just nominated as a visionary in the Gartner Magic Quadrant for Distributed File Systems and Object Storage. But according to Dave’s article, Swiftstack did not want to be “associated” with object storage that much, even though Swiftstack’s technology underpinning was all object storage. Strange.

Continue reading →

Cloud silos after eliminating silos

By cfheoh | September 20, 2017 - 1:10 pm |September 20, 2017 Amazon, Cisco, Cloud, Data Availability, Data Fabric, Data Management, Disaster Recovery, Elastifile, Filesystems, ILM, NetApp, Openstack, Software Defined Storage, Software-defined Datacenter, Virtualization, VMware

Can NetApp do it a bit better?

By cfheoh | March 11, 2017 - 11:21 pm |March 12, 2017 Acquisition, Amazon, Appliance, Backup, Business Continuity, Cloud, Data Archiving, Data Fabric, Data Management, Disaster Recovery, Elastifile, Gartner, Hyperconvergence, Interica, NetApp, Nimble Storage, NVMe, Object Storage, SNIA, Software Defined Storage, Software-defined Datacenter, Storage Market Share, Storage Tiering, Virtualization

1 Comment

[Preamble: I was a delegate of Storage Field Day 12. My expenses, travel and accommodation were paid for by GestaltIT, the organizer and I was not obligated to blog or promote the technologies presented in this event]

In Day 2 of Storage Field Day 12, I and the other delegates were hustled to NetApp’s Sunnyvale campus headquarters. That was a homecoming for me, and it was a bit ironic too.

Just 8 months ago, I was NetApp Malaysia Country Manager. That country sales lead role was my second stint with NetApp. I lasted almost 1 year.

17 years ago, my first stint with NetApp was the employee #2 in Malaysia as an SE. That SE stint went by quickly for 5 1/2 years, and I loved that time. Those Fall Classics NetApp used to have at the Batcave and the Fortress of Solitude left a mark with me, and the experiences still are as vivid as ever.

Despite what has happened in both stints and even outside the circle, I am still one of NetApp’s active cheerleaders in the Asia Pacific region. I even got accused by being biased as a community leader in the SNIA Malaysia Facebook page (unofficial but recognized by SNIA), because I was supposed to be neutral. I have put in 10 years to promote the storage technology community with SNIA Malaysia. [To the guy named Stanley, my response was be “Too bad, pick a religion“.]

The highlight of the SFD12 NetApp visit was of course, having lunch with Dave Hitz, one of the co-founders and the one still remaining. But throughout the presentations, I was unimpressed.

For me, the only one which stood out was CloudSync. I have read about CloudSync since NetApp Insight 2016 and yes, it’s a nice little piece of data shipping service between on-premise and AWS cloud.

Here’s how CloudSync looks like:

Continue reading →

Category Archives: Elastifile