Big Data Archives - Page 6 of 9

WekaIO controls their performance destiny

By cfheoh | March 17, 2019 - 5:33 pm |March 17, 2019 Amazon Web Services, Analytics, Appliance, Big Data, CIFS, Cloud, Deep Learning, Filesystems, Flash, High Performance Computing, Infiniband, Linux, Lustre, Machine Learning, Mellanox Technologies, NAS, NetApp, NFS, NVMe, Object Storage, PCIe, Performance Benchmark, Performance Caching, RDMA, Scale-out architecture, SMB, Software Defined Storage, Storage Field Day, Storage Optimization, Storage Tiering, Tech Field Day, Virtualization, WekaIO, Western Digital

3 Comments

[Preamble: I have been invited by GestaltIT as a delegate to their Tech Field Day for Storage Field Day 18 from Feb 27-Mar 1, 2019 in the Silicon Valley USA. My expenses, travel and accommodation were covered by GestaltIT, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

I was first introduced to WekaIO back in Storage Field Day 15. I did not blog about them back then, but I have followed their progress quite attentively throughout 2018. 2 Storage Field Days and a year later, they were back for Storage Field Day 18 with a new CTO, Andy Watson, and several performance benchmark records.

Blowout year

2018 was a blowout year for WekaIO. They have experienced over 400% growth, placed #1 in the Virtual Institute IO-500 10-node performance challenge, and also became #1 in the SPEC SFS 2014 performance and latency benchmark. (Note: This record was broken by NetApp a few days later but at a higher cost per client)

The Virtual Institute for I/O IO-500 10-node performance challenge was particularly interesting, because it pitted WekaIO against Oak Ridge National Lab (ORNL) Summit supercomputer, and WekaIO won. Details of the challenge were listed in Blocks and Files and WekaIO Matrix Filesystem became the fastest parallel file system in the world to date.

Control, control and control

I studied WekaIO’s architecture prior to this Field Day. And I spent quite a bit of time digesting and understanding their data paths, I/O paths and control paths, in particular, the diagram below:

Starting from the top right corner of the diagram, applications on the Linux client (running Weka Client software) and it presents to the Linux client as a POSIX-compliant file system. Through the network, the Linux client interacts with the WekaIO kernel-based VFS (virtual file system) driver which coordinates the Front End (grey box in upper right corner) to the Linux client. Other client-based protocols such as NFS, SMB, S3 and HDFS are also supported. The Front End then interacts with the NIC (which can be 10/100G Ethernet, Infiniband, and NVMeoF) through SR-IOV (single root IO virtualization), bypassing the Linux kernel for maximum throughput. This is with WekaIO’s own networking stack in user space. Continue reading →

Bridges to the clouds and more – NetApp NDAS

By cfheoh | March 15, 2019 - 7:48 am |March 15, 2019 Amazon Web Services, Analytics, API, Artificial Intelligence, Backup, Big Data, Cloud, Cohesity, Data Archiving, Data Availability, Data Fabric, Data Management, Data Protection, Data Security, Deep Learning, Disaster Recovery, Hyperconvergence, ILM, Machine Learning, NetApp, Reliability, Snapshots, Storage Field Day, Storage Tiering, Tech Field Day

2 Comments

The NetApp Data Fabric Vision

The NetApp Data Fabric vision has always been clear to me. Maybe it was because of my 2 stints with them, and I got well soaked in their culture. 3 simple points define the vision.

The Data Fabric is THE data singularity. Data can be anywhere – on-premises, the clouds, and more.
Have bridges, paths and workflows management to the Data, to move the data to wherever the data may be.
Work with technology partners to build tools and data systems to elevate the value of the data

That is how I see it. I wrote about the Transcendence of the Data Fabric vision 3+ years ago, and I emphasized the importance of the Data Pipeline in another NetApp blog almost a year ago. The introduction of NetApp Data Availability Services (NDAS) in the recently concluded Storage Field Day 18 was no different as NetApp constructs data bridges and paths to the AWS Cloud.

NetApp Data Availability Services

The NDAS feature is only available with ONTAP 9.5. With less than 5 clicks, data from ONTAP primary systems can be backed up to the secondary ONTAP target (running the NDAS proxy and the Copy to Cloud API), and then to AWS S3 buckets in the cloud.

Continue reading →

Clever Cohesity

By cfheoh | March 11, 2019 - 7:28 pm |March 11, 2019 Analytics, API, Appliance, Artificial Intelligence, Backup, Big Data, Business Continuity, Cloud, Clusters, Cohesity, Data, Data Archiving, Data Availability, Data Management, Data Protection, Data Security, Deep Learning, Disaster Recovery, Edge Computing, eDiscovery, Filesystems, Fog Computing, Hadoop, Hadoop Clusters, Hyperconvergence, Interica, Machine Learning, Object Storage, Scale-out architecture, Software Defined Storage, Storage Field Day, Tech Field Day, Veritas

3 Comments

This is clever. This is very smart.

The moment the Cohesity App Marketplace pitch was shared at the Storage Field Day 18 session, somewhere in my mind, enlightenment came to me.

The hyperconverged platform for secondary data, or is it?

When Cohesity came into the scene, they were branded the latest unicorn alongside Rubrik. Both were gunning for the top hyperconverged platform for secondary data. Crazy money was pouring into that segment – Cohesity got USD250 million in June 2018; Rubrik received USD261 million in Jan 2019 – making the market for hyperconverged platforms for secondary data red-hot. Continue reading →

Catch up (fast) – IBM Spectrum Protect Plus

By cfheoh | March 1, 2019 - 8:15 pm |March 1, 2019 Analytics, API, Backup, Big Data, Business Continuity, Cloud, Data Archiving, Data Availability, Data Management, Data Protection, Data Security, Deduplication, Disaster Recovery, eDiscovery, IBM, Object Storage, Reliability, Snapshots, Storage Field Day, Virtualization

5 Comments

The IBM Spectrum Protect Plus (SPP) team returned again for Storage Field Day 18, almost exactly 50 weeks when they introduced SPP to the Storage Field Day 15 delegates in 2018. My comments in my blog about IBM SPP were not flattering but the product was fairly new back then. I joined the other delegates to listen to IBM again this time around, and being open minded to listen and see their software upgrade.

Spectrum Protect Plus is NOT Spectrum Protect

First of all, it is important to call that IBM Spectrum Protect (SP)and IBM Spectrum Protect Plus (SPP) are 2 distinct products. The SP is the old Tivoli Storage Manager (TSM) while SPP is a more “modern” product, answering to virtualized environments and several public cloud service providers target platforms. To date, SP is version 8.1.x while SPP is introduced as version 10.1.4. There are “some” integration between SP and SPP, where SPP data can be “offloaded” to the SP platform for long term retention.

For one, I certainly am confused about IBM’s marketing and naming of both products, and I am sure many face the same predicament too. Continue reading →

VAST Data must be something special

By cfheoh | February 28, 2019 - 9:59 pm |March 1, 2019 Analytics, Artificial Intelligence, Big Data, CIFS, Cloud, Clusters, Composable Infrastructure, Data, Data Fabric, Data Management, Data Protection, Deduplication, Edge Computing, Filesystems, High Performance Computing, Infiniband, Machine Learning, NAS, NFS, NVMe, Object Storage, Scale-out architecture, Software Defined Storage, Tech Field Day, Vast Data, XtremIO

3 Comments

Vast Data coming out bash!

The delegates of Storage Field Days were always the lucky bunch. We have witnessed several storage technology companies coming out of stealth at these Tech Field Days. The recent ones in memory for me were Excelero and Hammerspace. But to have one where the venerable storage doyen, Mr. Howard Marks, Vast Data new tech evangelist, to introduce the deep dive of Vast Data technology was something special.

For those who knew Howard, he is fiercely independent, very storage technology smart, opinionated and not easily impressed. As a storage technology connoisseur myself, I believe Howard must have seen something special in Vast Data. They must be doing something extremely unique and impressive that someone like Howard could not resist, and made him jump to the vendor side. This sets the tone of my blog.

Continue reading →

Minio – the minimalist object storage technology

By cfheoh | February 7, 2019 - 10:57 am |February 7, 2019 Amazon Web Services, Analytics, API, Appliance, Big Data, Cloud, Data, Data Archiving, Data Fabric, Data Management, Data Protection, Data Security, Deep Learning, Disaster Recovery, Edge Computing, Filesystems, Flash, FreeNAS, HDS, High Performance Computing, Hitachi Vantara, Industry 4.0, Katana Logic, Linux, Machine Learning, Minio, NAS, NetApp, NFS, Object Storage, Openstack, Redhat, Scale-out architecture, Tech Field Day

4 Comments

The Marie Kondo Konmari fever is sweeping the world. Her decluttering and organizing the home methods are leading to a new way of life – Minimalism.

Complicated Storage Experience

Storage technology and its architecture are complex. We layer upon layer of abstraction and virtualization into storage design until at some stage, choke points lead to performance degradation, and management becomes difficult.

I recalled a particular training I attended back in 2006. I just joined Hitachi Data Systems for the Shell GUSto project. I was in Baltimore for the Hitachi NAS course. This was not their HNAS (their BlueArc acquisition) but their home grown NAS based on Linux. In the training, we were setting up NFS service. There were 36 steps required to setup and provision NFS and if there was a misstep, you start from the first command again. Coming from NetApp at the time, it was horrendous. NetApp ONTAP NFS setup and provisioning probably took 3 commands, and this Hitachi NAS setup and configuration was so much more complex. In the end, the experience was just unworldly for me.

Introducing Minio to my world, to Malaysia

Continue reading →

Sleepless in Malaysia with Object Storage

By cfheoh | January 22, 2019 - 1:42 pm |January 22, 2019 Amazon, Amazon Web Services, Analytics, API, Big Data, Cloud, Cloudian, Clusters, Data Management, Deep Learning, DellEMC, Dropbox, Filesystems, Google, Hadoop, Hadoop Clusters, HDS, IDC, IoT, iSCSI, Linux, Machine Learning, Microsoft, Minio, NAS, NFS, Object Storage, OpenIO, Redhat, Security, swiftstack

Object Storage? What’s that?

For the past couple of months, I have been speaking with a few parties in Malaysia about object storage technology. And I was fairly surprised with the responses.

The 2 reports

For a start, I did not set out to talk about object storage. It kind of fell onto my lap. 2 recent Hitachi Vantara reports revealed that countries like Australia, Hong Kong and even South East Asian countries were behind in their understanding of what object storage was, and the benefits it brought to the new generation of web scale and enterprise applications.

In the first report, an IDC survey sponsored by Hitachi Vantara, mentioned that 41% of the enterprises in Australia are not aware of object storage technology. In a similar survey, this one pointing towards Hong Kong and China, the percentages were 38% and 35% respectively. I would presume that the percentages for countries in South East Asia would not fall too far from the apple tree.

How is Malaysia doing?

However, I worry that the percentage number could be far more dire in Malaysia. In the past 2 months, responses from several conversations painted a darker hue about object storage technology with the companies in Malaysia. These included a reasonable sized hosting company, a well-established systems integrator, a software development company, several storage practitioners in Openstack and a DellEMC’s regional consultant for unstructured data. The collective conclusion was object storage technology was relatively unknown (probably similar to the percentages to the IDC/Hitachi Vantara reports), but it appeared to be shunned at this juncture. In web scale applications, Redhat Ceph block and files appeared popular in contrast to Openstack Swift. In enterprise applications, it was a toss of iSCSI and NFS.

Image from https://zdnet4.cbsistatic.com/hub/i/r/2018/04/24/c79e9dfb-b4a9-46bb-b831-f2c57fdf8a1d/resize/470xauto/5e4846d1bc7a034c382baf6dcbb612ed/cloud-storage.jpg

Continue reading →

Quantum Corp should spin off Stornext

By cfheoh | January 7, 2019 - 8:27 am |January 7, 2019 Acquisition, Analytics, API, Appliance, Artificial Intelligence, Backup, Big Data, Cloud, Commvault, Data, Data Archiving, Data Availability, Data Corruption, Data Domain, Data Fabric, Data Management, Data Protection, Data Security, Deduplication, Dell, DellEMC, Disaster Recovery, eDiscovery, Filesystems, High Performance Computing, Hyperconvergence, NetApp, Object Storage, Panasas, Pure Storage, Quantum Corporation, Scale-out architecture, Software Defined Storage, Stornext

1 Comment

What’s happening at Quantum Corporation?

I picked up the latest development news about Quantum Corporation. Last month, in December 2018, they secured a USD210 million financial lifeline to support their deflating business and their debts. And if you follow their development, they are with their 3rd CEO in the past 12 months, which is quite extraordinary. What is happening at Quantum Corp?

Quantum Logo (PRNewsFoto/Quantum Corp.)

Stornext – The Swiss Army knife of Data Management

I have known Quantum since 2000, very focused on the DLT tape library business. At that time, prior to the coming of LTO, DLT and its successor, SuperDLT dominated the tape market together with IBM. In 2006, they acquired ADIC, another tape vendor and became one of the largest tape library vendors in the world. From the ADIC acquisition, Quantum also got their rights on Stornext, a high performance scale out file system. I was deeply impressed with Stornext, and I once called it the Swiss Army knife of Data Management. The versatility of Stornext addressed many of the required functions within the data management lifecycle and workflows, and thus it has made its name in the Media and Entertainment space.

Jack of all trades, master of none

However, Quantum has never reached great heights in my opinion. They are everything to everybody, like a Jack of all trades, master of none. They are backup with their tape libraries and DXi series, archive and tiering with the Lattus, hybrid storage with QXS, and file system and scale-out with Stornext. If they have good business run rates and a healthy pipeline, having a broad product line is fine and dandy. But Quantum has been having CEO changes like turning a turnstile, and amid “a few” accounting missteps and a 2018 CEO who only lasted 5 months, they better steady their rocking boat quickly. Continue reading →

Storage and Data Management Planning crucial for Malaysian SMBs

By cfheoh | January 2, 2019 - 8:30 am |January 2, 2019 Backup, Big Data, Business Continuity, Cloud, Data, Data Availability, Data Management, Data Protection, Data Security, Disaster Recovery, High Performance Computing, Hyperconvergence, Machine Learning, NVMe, Performance Caching, Reliability, Scale-out architecture, Server SAN, Software Defined Storage, Software-defined Datacenter

Hybrid IT for 2019 and beyond

2019 is here.

I am especially buoyed by the strong network storage industry footing in 2018, reported by The Register last week. 2018 was certainly a blowout year for storage infrastructure and storage software, both for on-premises and the cloud computing platforms. The AWS Outposts announcement over a month ago also just affirmed that the new world is Hybrid IT. And there is plenty to look forward to in 2019.

Malaysian Economic Doldrums

Things are not as rosy for the Malaysia economy in 2019. It will be a challenging 2019 as reported by the Edge, a local business publication. The GDP (gross domestic product) of the first half of 2018 shrunk, from 5.9% in 2017, to 4.65%, and it is estimated to be 4.9% in 2019. With an inexperienced new government, a weak currency, and more competitive economies emerging in ASEAN, Malaysia small and medium businesses (SMBs) could be challenged.

The knee jerk reaction would be to cut the IT spending and revert to buying on price. This has happened too often, because there are always other operating costs that may be more pressing. Furthermore, many of the SMBs are still aimless when it comes to transforming their businesses into the digital data era, groping in the dark and sputtering to get its worth with their IT investments. Often, many are misinformed and stumbled, resulting in much higher wastage and costs.

There is a local saying here:

“Good thing No Cheap; Cheap thing No Good“

And the saying is very apt to describe that there is value in investing well, and the price factor should not always be the main determinant criteria of buying IT infrastructure, software and services.

Many of these SMBs also lack experienced IT staff to manage their IT environment. There is also a hurried urgency to modernize IT, because a well-planned and executed IT strategy and operations would definitely increase their Competitive Advantage. Continue reading →

From the past to the future

By cfheoh | December 17, 2018 - 10:17 am |December 17, 2018 100Gigabit Ethernet, Acquisition, Analytics, API, Appliance, Artificial Intelligence, Big Data, Business Continuity, Clusters, Composable Infrastructure, Data Archiving, Data Availability, Data Corruption, Data Fabric, Data Management, Data Protection, Data Security, Deduplication, Deep Learning, Dell, DellEMC, Disaster Recovery, Falconstor, FCoE, Fibre Channel, Filesystems, High Performance Computing, Hyperconvergence, Infiniband, Machine Learning, Memory Cloud, NAS, Nutanix, Performance Benchmark, Performance Caching, Scale-out architecture, Server SAN, Software Defined Storage, Software-defined Datacenter

The evolution of the Data Services Platform

Late 2017, I blogged about the Data Services Platform. Storage is no longer the storage infrastructure we know but has evolved to a platform where a plethora of data services are served. The changing face of storage is continually evolving as the IT industry changes. I take this opportunity to reflect what I wrote since I started blogging years ago, and look at the articles that are shaping up the landscape today and also some duds.

Some good ones …

One of the most memorable ones is about memory cloud. I wrote the article when Dell acquired a small company by the name of RNA Networks. I vividly recalled what was going through my mind when I wrote the blog. With the SAN, NAS and DAS, and even FAN (File Area Network) happening during that period, the first thing was the System Area Network, the original objective Infiniband and RDMA. I believed the final pool of where storage will be is the memory, hence I called it the “The Last Bastion – Memory“. RNA’s technology became part of Dell Fluid Architecture.

True enough, the present technology of Storage Class Memory and SNIA’s NVDIMM are along the memory cloud I espoused years ago.

What about Fibre Channel over Ethernet (FCoE)? It wasn’t a compelling enough technology for me when it came into the game. Reduced port and cable counts, and reduced power consumption were what the FCoE folks were pitching, but the cost of putting in the FC switches, the HBAs were just too great as an investment. In the end, we could see the cracks of the FCoE story, and I wrote the pre-mature eulogy of FCoE in my 2012 blog. I got some unsavoury comments writing that blog back then, but fast forward to the present, FCoE isn’t a force anymore.

Weeks ago, Amazon Web Services (AWS) just became a hybrid cloud service provider/vendor with the Outposts announcement. It didn’t surprise me but it may have shook the traditional systems integrators. I took the stance 2 years ago when AWS partnered with VMware and juxtaposed it to the philosophical quote in the 1993 Jurassic Park movie – “Life will not be contained, … Life finds a way“.

Continue reading →

Category Archives: Big Data