100Gigabit Ethernet Archives - Page 2 of 2

Is General Purpose Object Storage disenfranchised?

By cfheoh | December 23, 2019 - 5:40 pm |January 14, 2020 100Gigabit Ethernet, Amazon Web Services, Analytics, API, Artificial Intelligence, Big Data, BYOD, Ceph, Cloud, Cloudian, Clusters, Deep Learning, DellEMC, Docker, Dropbox, Edge Computing, Filesystems, Flash, Gartner, Hadoop, HDS, High Performance Computing, Hitachi Vantara, IDC, Industry 4.0, IoT, Lustre, Machine Learning, Mellanox Technologies, Minio, NetApp, Object Storage, OpenIO, Openstack, Performance Benchmark, Reliability, Scale-out architecture, Software Defined Storage, Storage Field Day, Storage Market Share, swiftstack, Tape storage, Tech Field Day

6 Comments

[Disclosure: I am invited by GestaltIT as a delegate to their Storage Field Day 19 event from Jan 22-24, 2020 in the Silicon Valley USA. My expenses, travel, accommodation and conference fees will be covered by GestaltIT, the organizer and I am not obligated to blog or promote the vendors’ technologies to be presented at this event. The content of this blog is of my own opinions and views]

This is NOT an advertisement for coloured balls.

This is the license to brag for the vendors in the next 2 weeks or so, as we approach the 2020 new year. This, of course, is the latest 2019 IDC Marketscape for Object-based Storage, released last week.

My object storage mentions

I have written extensively about Object Storage since 2011. With different angles and perspectives, here are some of them:

The Future is Intelligent Objects (2011)
What should be Cloud Storage? (2011)
APIs that stick in Storage (2012)
Has Object Storage become the Everything Store? (2013)
Of Object Storage, Filesystems and Multicloud (2017)
My Dilemma of Stateful Storage Marriage (2018)
The Malaysian Openstack Storage Conundrum (2018)
Sleepless in Malaysia with Object Storage (2019)
The Waning Light of Openstack Swift (2019)

Continue reading →

Storage Performance Considerations for AI Data Paths

By cfheoh | June 17, 2019 - 10:50 am |June 17, 2019 100Gigabit Ethernet, Algorithm, Analytics, API, Artificial Intelligence, Big Data, Cloud, Composable Infrastructure, Data, Data Fabric, Data Management, Data Privacy, Data Security, Digital Transformation, Drivescale, E8 Storage, Edge Computing, Elastifile, Excelero, Filesystems, High Performance Computing, Hyperconvergence, Industry 4.0, Infiniband, Intel, Liqid, Lustre, Machine Learning, Mellanox Technologies, NVMe, Object Storage, Performance Benchmark, Performance Caching, Quantum Corporation, RDMA, Software-defined Datacenter, Storage Optimization, Storage Tiering, ThinkParq, Vast Data, Virtualization, WekaIO

1 Comment

The hype of Deep Learning (DL), Machine Learning (ML) and Artificial Intelligence (AI) has reached an unprecedented frenzy. Every infrastructure vendor from servers, to networking, to storage has a word to say or play about DL/ML/AI. This prompted me to explore this hyped ecosystem from a storage perspective, notably from a storage performance requirement point-of-view.

One question on my mind

There are plenty of questions on my mind. One stood out and that is related to storage performance requirements.

Reading and learning from one storage technology vendor to another, the context of everyone’s play against their competitors seems to be “They are archaic, they are legacy. Our architecture is built from ground up, modern, NVMe-enabled“. And there are more juxtaposing, but you get the picture – “We are better, no doubt“.

Are the data patterns and behaviours of AI different? How do they affect the storage design as the data moves through the workflow, the data paths and the lifecycle of the AI ecosystem?

Continue reading →

Scaling new HPC with Composable Architecture

By cfheoh | May 10, 2019 - 4:50 pm |May 10, 2019 100Gigabit Ethernet, Analytics, API, Appliance, Artificial Intelligence, Big Data, Cloud, Clusters, Composable Infrastructure, Containers, Data Fabric, Data Management, Deep Learning, DellEMC, Drivescale, High Performance Computing, Hyperconvergence, Infiniband, Liqid, Machine Learning, nVidia, NVMe, PCIe, RDMA, Scale-out architecture, Software Defined Storage, Software-defined Datacenter, Tech Field Day, Unified Storage, Virtualization

2 Comments

[Disclosure: I was invited by Dell Technologies as a delegate to their Dell Technologies World 2019 Conference from Apr 29-May 1, 2019 in the Las Vegas USA. Tech Field Day Extra was an included activity as part of the Dell Technologies World. My expenses, travel, accommodation and conference fees were covered by Dell Technologies, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

Deep Learning, Neural Networks, Machine Learning and subsequently Artificial Intelligence (AI) are the new generation of applications and workloads to the commercial HPC systems. Different from the traditional, more scientific and engineering HPC workloads, I have written about the new dawn of supercomputing and the attractive posture of commercial HPC.

Don’t be idle

From the business perspective, the investment of HPC systems is high most of the time, and justifying it to the executives and the investors is not easy. Therefore, it is critical to keep feeding the HPC systems and significantly minimize the idle times for compute, GPUs, network and storage.

However, almost all HPC systems today are inflexible. Once assigned to a project, the resources pretty much stay with the project, even when the workload processing of the project is idle and waiting. Of course, we have to bear in mind that not all resources are fully abstracted, virtualized and software-defined whereby you can carve out pieces of the hardware and deliver a percentage of that resource. Case in point is the CPU, where you cannot assign certain clock cycles of CPU to one project and another half to the other. The technology isn’t there yet. Certain resources like GPU is going down the path of Virtual GPU, and into the realm of resource disaggregation. Eventually, all resources of the HPC systems – CPU, memory, FPGA, GPU, PCIe channels, NVMe paths, IOPS, bandwidth, burst buffers etc – should be disaggregated and pooled for disparate applications and workloads based on demands of usage, time and performance.

Hence we are beginning to see the disaggregated HPC systems resources composed and built up the meet the diverse mix and needs of HPC applications and workloads. This is even more acute when a AI project might grow cold, but the training of AL/ML/DL workloads continues to stay hot

Liqid the early leader in Composable Architecture

Continue reading →

StorPool – Block storage managed well

By cfheoh | March 13, 2019 - 8:34 am |March 13, 2019 100Gigabit Ethernet, API, Cloud, Clusters, Data Availability, Data Management, Disks, Filesystems, High Performance Computing, iSCSI, Linux, Lustre, Nexenta, NVMe, Openstack, Performance Benchmark, Performance Caching, Scale-out architecture, Server SAN, Software Defined Storage, Storage Field Day, Storage Optimization, Storpool, Tech Field Day, Virtualization, VMware

2 Comments

[Preamble: I have been invited by GestaltIT as a delegate to their Tech Field Day for Storage Field Day 18 from Feb 27-Mar 1, 2019 in the Silicon Valley USA. My expenses, travel and accommodation were covered by GestaltIT, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

Storage technology is complex. Storage infrastructure and data management operations are not trivial, despite what the hyperscalers like Amazon Web Services and Microsoft Azure would like you to think. As the adoption of cloud infrastructure services grow, the small and medium businesses/enterprises (SMB/SME) are usually left to their own devices to manage the virtual storage infrastructure. Cloud Service Providers (CSPs) addressing the SMB/SME market are looking for easier, worry-free, software-defined storage to elevate their value to their customers.

Managed high performance block storage

Enter StorPool.

StorPool is a scale-out block storage technology, capable of delivering 1 million+ IOPS with sub-milliseconds response times. As described by fellow delegate, Ray Lucchesi in his recent blog, they were able to achieve these impressive performance numbers in their demo, without the high throughput RDMA network or the storage class memory of Intel Optane. Continue reading →

Microsoft desires Mellanox

By cfheoh | December 20, 2018 - 11:02 am |December 20, 2018 100Gigabit Ethernet, Acquisition, Amazon, Artificial Intelligence, Cloud, Data Fabric, Data Management, Deep Learning, Edge Computing, High Performance Computing, Infiniband, Machine Learning, Mellanox Technologies, Microsoft, NVMe, Storage Field Day, Tech Field Day, Virtualization

From the past to the future

By cfheoh | December 17, 2018 - 10:17 am |December 17, 2018 100Gigabit Ethernet, Acquisition, Analytics, API, Appliance, Artificial Intelligence, Big Data, Business Continuity, Clusters, Composable Infrastructure, Data Archiving, Data Availability, Data Corruption, Data Fabric, Data Management, Data Protection, Data Security, Deduplication, Deep Learning, Dell, DellEMC, Disaster Recovery, Falconstor, FCoE, Fibre Channel, Filesystems, High Performance Computing, Hyperconvergence, Infiniband, Machine Learning, Memory Cloud, NAS, Nutanix, Performance Benchmark, Performance Caching, Scale-out architecture, Server SAN, Software Defined Storage, Software-defined Datacenter

The evolution of the Data Services Platform

Late 2017, I blogged about the Data Services Platform. Storage is no longer the storage infrastructure we know but has evolved to a platform where a plethora of data services are served. The changing face of storage is continually evolving as the IT industry changes. I take this opportunity to reflect what I wrote since I started blogging years ago, and look at the articles that are shaping up the landscape today and also some duds.

Some good ones …

One of the most memorable ones is about memory cloud. I wrote the article when Dell acquired a small company by the name of RNA Networks. I vividly recalled what was going through my mind when I wrote the blog. With the SAN, NAS and DAS, and even FAN (File Area Network) happening during that period, the first thing was the System Area Network, the original objective Infiniband and RDMA. I believed the final pool of where storage will be is the memory, hence I called it the “The Last Bastion – Memory“. RNA’s technology became part of Dell Fluid Architecture.

True enough, the present technology of Storage Class Memory and SNIA’s NVDIMM are along the memory cloud I espoused years ago.

What about Fibre Channel over Ethernet (FCoE)? It wasn’t a compelling enough technology for me when it came into the game. Reduced port and cable counts, and reduced power consumption were what the FCoE folks were pitching, but the cost of putting in the FC switches, the HBAs were just too great as an investment. In the end, we could see the cracks of the FCoE story, and I wrote the pre-mature eulogy of FCoE in my 2012 blog. I got some unsavoury comments writing that blog back then, but fast forward to the present, FCoE isn’t a force anymore.

Weeks ago, Amazon Web Services (AWS) just became a hybrid cloud service provider/vendor with the Outposts announcement. It didn’t surprise me but it may have shook the traditional systems integrators. I took the stance 2 years ago when AWS partnered with VMware and juxtaposed it to the philosophical quote in the 1993 Jurassic Park movie – “Life will not be contained, … Life finds a way“.

Continue reading →

Sexy HPC storage is all the rage

By cfheoh | November 26, 2018 - 10:44 am |November 26, 2018 100Gigabit Ethernet, Analytics, API, Artificial Intelligence, BeeGFS, CIFS, Clusters, Data Management, Deep Learning, DellEMC, Disks, E8 Storage, EMC, Excelero, Filesystems, Hadoop Clusters, High Performance Computing, Hyperconvergence, IBM, Infiniband, Intel, Linux, Lustre, Machine Learning, Mellanox, Memory Cloud, NAS, NetApp, NFS, Panasas, Performance Benchmark, Performance Caching, Pure Storage, RDMA, Scale-out architecture, SMB, Software-defined Datacenter, Storage Field Day, Tech Field Day, ThinkParq, WekaIO

HPC is sexy

There is no denying it. HPC is sexy. HPC Storage is just as sexy.

Looking at the latest buzz from Super Computing Conference 2018 which happened in Dallas 2 weeks ago, the number of storage related vendors participating was staggering. Panasas, Weka.io, Excelero, BeeGFS, are the ones that I know because I got friends posting their highlights. Then there are the perennial vendors like IBM, Dell, HPE, NetApp, Huawei, Supermicro, and so many more. A quick check on the SC18 website showed that there were 391 exhibitors on the floor.

And this is driven by the unrelentless demand for higher and higher performance of computing, and along with it, the demands for faster and faster storage performance. Commercialization of Artificial Intelligence (AI), Deep Learning (DL) and newer applications and workloads together with the traditional HPC workloads are driving these ever increasing requirements. However, most enterprise storage platforms were not designed to meet the demands of these new generation of applications and workloads, as many have been led to believe. Why so?

I had a couple of conversations with a few well known vendors around the topic of HPC Storage. And several responses thrown back were to put Flash and NVMe to solve the high demands of HPC storage performance. In my mind, these responses were too trivial, too irresponsible. So I wanted to write this blog to share my views on HPC storage, and not just about its performance.

The HPC lines are blurring

I picked up this video (below) a few days ago. It was insideHPC Rich Brueckner interview with Dr. Goh Eng Lim, HPE CTO and renowned HPC expert about the convergence of both traditional and commercial HPC applications and workloads.

I liked the conversation in the video because it addressed the 2 different approaches. And I welcomed Dr. Goh’s invitation to the Commercial HPC community to work with the Traditional HPC vendors to help push the envelope towards Exascale SuperComputing.

Continue reading →

Disaggregation or hyperconvergence?

By cfheoh | October 31, 2018 - 12:39 pm |October 31, 2018 100Gigabit Ethernet, Analytics, API, Clusters, Composable Infrastructure, Data Fabric, Datrium, Disks, DMTF, Drivescale, High Performance Computing, Hyperconvergence, Intel, iSCSI, Machine Learning, NetApp, Openstack, SNIA, Software Defined Storage, Software-defined Datacenter, Tech Field Day, Western Digital

1 Comment

[Preamble: I have been invited by GestaltIT as a delegate to their TechFieldDay from Oct 17-19, 2018 in the Silicon Valley USA. My expenses, travel and accommodation are covered by GestaltIT, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

There is an argument about NetApp‘s HCI (hyperconverged infrastructure). It is not really a hyperconverged product at all, according to one school of thought. Maybe NetApp is just riding on the hyperconvergence marketing coat tails, and just wanted to be associated to the HCI hot streak. In the same spectrum of argument, Datrium decided to call their technology open convergence, clearly trying not to be related to hyperconvergence.

Hyperconvergence has been enjoying a period of renaissance for a few years now. Leaders like Nutanix, VMware vSAN, Cisco Hyperflex and HPE Simplivity have been dominating the scene, and touting great IT benefits and eliminating IT efficiencies. But in these technologies, performance and capacity are tightly intertwined. That means that in each of the individual hyperconverged nodes, typically starting with a trio of nodes, the processing power and the storage capacity comes together. You have to accept both resources as a node. If you want more processing power, you get the additional storage capacity that comes with that node. If you want more storage capacity, you get more processing power whether you like it or not. This means, you get underutilized resources over time, and definitely not rightsized for the job.

And here in Malaysia, we have seen vendors throw in hyperconverged infrastructure solutions for every single requirement. That was why I wrote a piece about some zealots of hyperconverged solutions 3+ years ago. When you think you have a magical hammer, every problem is a nail. 😉

In my radar, NetApp and Datrium are the only 2 vendors that offer separate nodes for compute processing and storage capacity and still fall within the hyperconverged space. This approach obviously benefits the IT planners and the IT architects, and the customers too because they get what they want for their business. However, the disaggregation of compute processing and storage leads to the argument of whether these 2 companies belong to the hyperconverged infrastructure category.

Continue reading →

The Network is Still the Computer

By cfheoh | October 22, 2018 - 11:35 am |October 22, 2018 100Gigabit Ethernet, 10Gigabit Ethernet, Analytics, API, Artificial Intelligence, Big Data, Cisco, Clusters, Data Management, Deep Learning, Disks, Drivescale, Fibre Channel, Filesystems, Hadoop, Hadoop Clusters, High Performance Computing, Infiniband, iSCSI, Linux, Machine Learning, MapReduce, NFS, NVMe, PCIe, Performance Benchmark, RDMA, Scale-out architecture, Storage Field Day, Tech Field Day, Virtualization

3 Comments

Sun Microsystems coined the phrase “The Network is the Computer“. It became one of the most powerful ideologies in the computing world, but over the years, many technology companies have tried to emulate and practise the mantra, but fell short.

I have never heard of Drivescale. It wasn’t in my radar until the legendary NFS guru, Brian Pawlowski joined them in April this year. Beepy, as he is known, was CTO of NetApp and later at Pure Storage, and held many technology leadership roles, including leading the development of NFSv3 and v4.

Prior to Tech Field Day 17, I was given some “homework”. Stephen Foskett, Chief Cat Herder (as he is known) of Tech Field Days and Storage Field Days, highly recommended Drivescale and asked the delegates to pick up some notes on their technology. Going through a couple of the videos, Drivescale’s message and philosophy resonated well with me. Perhaps it was their Sun Microsystems DNA? Many of the Drivescale team members were from Sun, and I was previously from Sun as well. I was drinking Sun’s Kool Aid by the bucket loads even before I graduated in 1991, and so what Drivescale preached made a lot of sense to me.Drivescale is all about Scale-Out Architecture at the webscale level, to address the massive scale of data processing. To understand deeper, we must think about “Data Locality” and “Data Mobility“. I frequently use these 2 “points of discussion” in my consulting practice in architecting and designing data center infrastructure. The gist of data locality is simple – the closer the data is to the processing, the cheaper/lightweight/efficient it gets. Moving data – the data mobility part – is expensive.

Continue reading →

Category Archives: 100Gigabit Ethernet