Object Storage Archives - Page 8 of 10

Pure Electric!

By cfheoh | July 20, 2017 - 10:31 am |July 20, 2017 Analytics, API, Appliance, Backup, Big Data, Business Continuity, CIFS, Data Availability, Data Fabric, Data Management, Filesystems, Flash, Gartner, Microsoft, NAS, NVMe, Object Storage, Pure Storage, Reliability, Snapshots, Software Defined Storage, Software-defined Datacenter, Solid State Devices, Storage Tiering, Virtualization

The changing face of storage

By cfheoh | April 27, 2017 - 11:25 am |April 27, 2017 Analytics, Backup, Big Data, Business Continuity, Data, Data Archiving, Data Availability, Data Management, Disaster Recovery, Disks, Filesystems, High Performance Computing, ILM, Object Storage, Scale-out architecture, Security, Software Defined Storage, Software-defined Datacenter, Storage Optimization, Storage Tiering, Virtualization

**“No, we are not a storage company anymore. We are a data management company now.“**

I was reading a Forbes article interviewing NetApp’s CIO, Bill Miller. It was titled:

NetApp’s CIO Helps Drive Company’s Shift From Data Storage To Data Management

I was fairly surprised about the time it took for that mindset shift messaging from storage to data management. I am sure that NetApp has been doing that for years internally.

To me, the writing has been in the wall for years. But weak perception of storage, at least in this part of Asia, still lingers as that clunky, behind the glassed walls and crufty closets, noisy box of full of hard disk drives lodged with snakes and snakes of orange, turquoise or white cables. 😉

The article may come as a revelation to some, but the world of storage has changed indefinitely. The blurring of the lines began when software defined storage, or even earlier in the form of storage virtualization, took form. I even came up with my definition a couple of years ago about the changing face of storage framework. Instead of calling it data management, I called the new storage framework, the Data Services Platform.

So, this is my version of the storage technology platform of today. This is the Data Services Platform I have been touting to many for the last couple of years. It is not just storage technology anymore; it is much more than that.

Continue reading →

Can NetApp do it a bit better?

By cfheoh | March 11, 2017 - 11:21 pm |March 12, 2017 Acquisition, Amazon, Appliance, Backup, Business Continuity, Cloud, Data Archiving, Data Fabric, Data Management, Disaster Recovery, Elastifile, Gartner, Hyperconvergence, Interica, NetApp, Nimble Storage, NVMe, Object Storage, SNIA, Software Defined Storage, Software-defined Datacenter, Storage Market Share, Storage Tiering, Virtualization

1 Comment

[Preamble: I was a delegate of Storage Field Day 12. My expenses, travel and accommodation were paid for by GestaltIT, the organizer and I was not obligated to blog or promote the technologies presented in this event]

In Day 2 of Storage Field Day 12, I and the other delegates were hustled to NetApp’s Sunnyvale campus headquarters. That was a homecoming for me, and it was a bit ironic too.

Just 8 months ago, I was NetApp Malaysia Country Manager. That country sales lead role was my second stint with NetApp. I lasted almost 1 year.

17 years ago, my first stint with NetApp was the employee #2 in Malaysia as an SE. That SE stint went by quickly for 5 1/2 years, and I loved that time. Those Fall Classics NetApp used to have at the Batcave and the Fortress of Solitude left a mark with me, and the experiences still are as vivid as ever.

Despite what has happened in both stints and even outside the circle, I am still one of NetApp’s active cheerleaders in the Asia Pacific region. I even got accused by being biased as a community leader in the SNIA Malaysia Facebook page (unofficial but recognized by SNIA), because I was supposed to be neutral. I have put in 10 years to promote the storage technology community with SNIA Malaysia. [To the guy named Stanley, my response was be “Too bad, pick a religion“.]

The highlight of the SFD12 NetApp visit was of course, having lunch with Dave Hitz, one of the co-founders and the one still remaining. But throughout the presentations, I was unimpressed.

For me, the only one which stood out was CloudSync. I have read about CloudSync since NetApp Insight 2016 and yes, it’s a nice little piece of data shipping service between on-premise and AWS cloud.

Here’s how CloudSync looks like:

Continue reading →

The engineering of Elastifile

By cfheoh | March 9, 2017 - 8:51 pm |March 9, 2017 10Gigabit Ethernet, Analytics, API, Big Data, CIFS, Cloud, Data Fabric, Data Management, Elastifile, Filesystems, Flash, Hyperconvergence, Linux, NAS, NFS, NVMe, Object Storage, Performance Caching, Reliability, Scale-out architecture, Software Defined Storage, Solid State Devices, Virtualization, XtremIO

2 Comments

When it comes to large scale storage capacity requirements with distributed cloud and on-premise capability, object storage is all the rage. Amazon Web Services started the object-based S3 storage service more than a decade ago, and the romance with object storage started.

Today, there are hundreds of object-based storage vendors out there, touting features after features of invincibility. But after researching and reading through many design and architecture papers, I found that many object-based storage technology vendors began to sound the same.

At the back of my mind, object storage is not easy when it comes to most applications integration. Yes, there is a new breed of cloud-based applications with RESTful CRUD API operations to access object storage, but most applications still rely on file systems to access storage for capacity, performance and protection.

These CRUD and CRUD-like APIs are the common semantics of interfacing object storage platforms. But many, many real-world applications do not have the object semantics to interface with storage. They are mostly designed to interface and interact with file systems, and secretly, I believe many application developers and users want a file system interface to storage. It does not matter if the storage is on-premise or in the cloud.

Let’s not kid ourselves. We are most natural when we work with files and folders.

Implementing object storage also denies us the ability to optimally utilize Flash and solid state storage on-premise when the compute is in the cloud. Similarly, when the compute is on-premise and the flash-based object storage is in the cloud, you get a mismatch of performance and availability requirements as well. In the end, there has to be a compromise.

Another “feature” of object storage is its poor ability to handle transactional data. Most of the object storage do not allow modification of data once the object has been created. Putting a NAS front (aka a NAS gateway) does not take away the fact that it is still object-based storage at the very core of the infrastructure, regardless if it is on-premise or in the cloud.

Resiliency, latency and scalability are the greatest challenges when we want to build a true globally distributed storage or data services platform. Object storage can be resilient and it can scale, but it has to compromise performance and latency to be so. And managing object storage will not be as natural as to managing a file system with folders and files.

Enter Elastifile.

Continue reading →

Let’s smoke the storage peace pipe

By cfheoh | September 29, 2016 - 12:07 pm |September 29, 2016 ATA over Ethernet, Cloud, Data, Data Fabric, Data Management, Disks, FCoE, Fibre Channel, Flash, High Performance Computing, Hyperconvergence, NAS, NetApp, NFS, NVMe, Object Storage, Openstack, PCIe, Performance Benchmark, Performance Caching, Reliability, Scale-out architecture, SCSI, Simplivity, SNIA, Software Defined Storage, Software-defined Datacenter, Solid State Devices, Storage Market Share, Storage Optimization, Storage Tiering, Tegile, Unified Storage, Virtualization

2 Comments

NVMe (Non-Volatile Memory Express) is upon us. And in the next 2-3 years, we will see a slew of new storage solutions and technology based on NVMe.

Just a few days ago, The Register released an article “Seventeen hopefuls fight for the NVMe Fabric array crown“, and it was timely. I, for one, cannot be more excited about the development and advancement of NVMe and the upcoming NVMeF (NVMe over Fabrics).

This is it. This is the one that will end the wars of DAS, NAS and SAN and unite the warring factions between server-based SAN (the sexy name differentiating old DAS and new DAS) and the networked storage of SAN and NAS. There will be PEACE.

Remember this?

Nutanix popularized the “No SAN” movement which later led to VMware VSAN and other server-based SAN solutions, hyperconverged techs such as PernixData (acquired by Nutanix), DataCore, EMC ScaleIO and also operated in hyperscalers – the likes of Facebook and Google. The hyperconverged solutions and the server-based SAN lines blurred of storage but still, they are not the usual networked storage architectures of SAN and NAS. I blogged about this, mentioning about how the pendulum has swung back to favour DAS, or to put it more appropriately, server-based SAN. There was always a “Great Divide” between the 2 modes of storage architectures. Continue reading →

Can CDMI emancipate an interoperable medical records cloud ecosystem?

By cfheoh | December 28, 2015 - 7:37 pm |December 28, 2015 Acquisition, API, Appliance, Cloud, Data Archiving, Data Management, ILM, Linux, NAS, NetApp, NFS, Object Storage, Scale-out architecture, SNIA, Software Defined Storage

Comments Off

PREFACE: This is just a thought, an idea. I am by no means an expert in this area. I have researched this to inspire a thought process of how we can bring together 2 disparate worlds of medical records and imaging with the emerging cloud services for healthcare.

Healthcare has been moving out of its archaic shell in the past few years, and digital healthcare technology and services are booming. And this movement is part of the digital transformation which could eventually lead to a secure and compliant distribution and collaboration of health data, medical imaging and electronic medical records (EMR).

It is a blessing that today’s medical imaging industry has been consolidated with the DICOM (Digital Imaging and Communications in Medicine) standard. DICOM dictates the how medical imaging information and pictures are used, stored, printed, transmitted and exchanged. It is also a communication protocol which runs over TCP/IP, and links up different service class providers (SCPs) and service class users (SCUs), and the backend systems such as PACS (Picture Archiving & Communications Systems) and RIS (Radiology Information Systems).

Another well accepted standard is HL7 (Health Level 7), a similar Layer 7, application-level communication protocol for transferring and exchanging clinical and administrative data.

The diagram below shows a self-contained ecosystem involving the front-end HIS (Hospital Information Systems), and the integration of healthcare, medical systems and other DICOM modalities.

(Picture courtesy of Meddiff Technologies)

Continue reading →

Oops, excuse me but your silo is showing

By cfheoh | April 8, 2015 - 8:07 am |April 10, 2015 Appliance, Backup, Cloud, Data, Data Archiving, Data Availability, Data Management, Filesystems, High Performance Computing, Hyperconvergence, Jon Toigo, NAS, NetApp, Nutanix, Object Storage, Performance Benchmark, Reliability, Scale-out architecture, Security, Simplivity, SNIA, Software Defined Storage, Software-defined Datacenter, Solid State Devices, Storage Tiering, Unified Storage, Virtualization, VMware

2 Comments

It is the morning that the SNIA Global Steering Committee reporting session is starting soon. I am in the office extremely early waiting for my turn to share the happenings in SNIA Malaysia.

And of late, I have been getting a lot of calls to catch up on hot technologies, notably All Flash Storage arrays and hyper-converged infrastructure. Even though I am now working for Interica, a company that focuses on Oil & Gas exploration and production software, my free coffee sessions with folks from the IT side have not diminished. And I recalled a week back in mid-March where I had coffee overdose!

Flash storage and hyperconvergence are HOT! Despite the hypes and frenzies of both flash storage and hyperconvergence, I still believe that integrating either or, or both, still have an effect that many IT managers overlook. The effect is a data silo.

Continue reading →

The reverse wars – DAS vs NAS vs SAN

By cfheoh | March 13, 2015 - 9:54 am |March 13, 2015 10Gigabit Ethernet, Appliance, ATA over Ethernet, Avere, CIFS, Cloud, Data, Disks, EMC, Fibre Channel, Filesystems, HDS, High Performance Computing, Hyperconvergence, iSCSI, Linux, Memory Cloud, Microsoft, NAS, NetApp, Nexenta, NFS, Nutanix, NVMe, Object Storage, Open Compute Project, Openstack, Panasas, PCIe, Performance Benchmark, Performance Caching, Scale-out architecture, SCSI, Seagate, Server SAN, Simplivity, SMB, Software Defined Storage, Solaris, Solid State Devices, Storage Optimization, Storage Tiering, Unified Storage, Virident, Virsto, Virtualization, VMware, XtremIO

4 Comments

It has been quite an interesting 2 decades.

In the beginning (starting in the early to mid-90s), SAN (Storage Area Network) was the dominant architecture. DAS (Direct Attached Storage) was on the wane as the channel-like throughput of Fibre Channel protocol coupled by the million-device addressing of FC obliterated parallel SCSI, which was only able to handle 16 devices and throughput up to 80 (later on 160 and 320) MB/sec.

NAS, defined by CIFS/SMB and NFS protocols – was happily chugging along the 100 Mbit/sec network, and occasionally getting sucked into the arguments about why SAN was better than NAS. I was already heavily dipped into NFS, because I was pretty much a SunOS/Solaris bigot back then.

When I joined NetApp in Malaysia in 2000, that NAS-SAN wars were going on, waiting for me. NetApp (or Network Appliance as it was known then) was trying to grow beyond its dot-com roots, into the enterprise space and guys like EMC and HDS were frequently trying to put NetApp down.

“It’s a toy…” was the most common jibe I got in regular engagements until EMC suddenly decided to attack Network Appliance directly with their EMC CLARiiON IP4700. EMC guys would fondly remember this as the “NetApp killer“. Continue reading →

Hail Hydra!

By cfheoh | November 20, 2014 - 3:40 pm |November 20, 2014 Actifio, Appliance, Backup, Big Data, CIFS, Data Management, Deduplication, EMC, Filesystems, HP, NAS, NFS, Object Storage, RAID, Reliability, Scale-out architecture, Virtualization

2 Comments

The last of the Storage Field Day 6 on November 7th took me and the other delegates to NEC. There was an obvious, yet eerie silence among everyone about this visit. NEC? Are you kidding me?

NEC isn’t exactly THE exciting storage company in the Silicon Valley, yet I was pleasantly surprised with their HydraStorprowess. It is indeed quite a beast, with published numbers of backup throughput of 4PB/hour, and scales to 100PB of capacity. Most impressive indeed, and HydraStor deserves this blogger’s honourable architectural dissection.

HydraStor is NEC’s grid-based, scale-out storage platform with an object storage backend. The technology, powered by the DynamicStor ™ software, a distributed file system laid over the HydraStor grid architecture. At the same time, it has the DataRedux™ technology that provides the global in-line deduplication as the HydraStor ingests data for data protection, replication, archiving and WORM purposes. It is a massive data consolidation platform, storing gazillion loads of data (100PB you say?) for short-term and long-term retention and recovery.

The architecture is indeed solid, and its data availability goes beyond traditional RAID-level resiliency. HydraStor employs their proprietary erasure coding, called Distributed Resilient Data™. The resiliency knob can be configured to withstand 6 concurrent disks or nodes failure, but by default configured with a resiliency level of 3.

We can quickly deduce that DynamicStor™, DataRedux™ and Distributed Resilient Data™ are the technology pillars of HydraStor. How do they work, and how do they work together?

Let’s look a bit deeper into the HydraStor architecture.

HydraStor is made up of 2 types of nodes:

Accelerator Nodes
Storage Nodes

The Accelerator Nodes (AN) are the access nodes. They interface with the HydraStor front end, which could be CIFS, NFS or OST (Open Storage Technology). The AN nodes chunks the in-coming data and performs in-line deduplication at a very high speed. It can reach speed of 300TB/hour, which is blazingly fast!

The AN nodes also runs DynamicStor™, handling the performance heavy-lifting portion of HydraStor. The chunked data from the AN nodes are then passed on to the Storage Nodes (SN), where they are further “deduped in-line” to determined if the chunks are unique or not. It is a two-step inline deduplication process. Below is a diagram showing the ANs built above the SNs in the HydraStor grid architecture.

The HydraStor grid architecture is also a very scalable architecture, allow the dynamic scale-in and scale-out of both ANs and SNs. AN nodes and SN nodes can be added or removed into the system, auto-configuring and auto-optimizing while everything stays online. This capability further strengthens the reliability and the resiliency of the HydraStor.

Moving on to DataRedux™. DataRedux™ is HydraStor’s global in-line data deduplication technology. It performs dedupe at the sub-file level, with variable length window. This is performed at the AN nodes and the SN nodes level,chunking and creating unique hash values. All unique chunks are further compressed with a modified LZ compression algorithm, shrinking the data to its optimized footprint on the disk storage. To maintain the global in-line deduplication, the hash table is available across the HydraStor cluster.

The unique data chunk resulting from deduplication and compression are then written to disks using the configured Distributed Resilient Data™ (DRD) algorithm, at its set resiliency level.

At the junction of DRD, with erasure coding parity, the data is broken up into multiples of fragments and assigned a parity to a grouping of fragments. If the resiliency level is set to 3 (the default), the data is broken into 12 pieces, 9 data fragments + 3 parity fragments. The 3 parity fragments corresponds to the resiliency level of 3. See diagram below of the 12 fragments spread across a group of selected disks in the storage pool of the Storage Nodes.

If the HydraStor experiences a failure in the disks or nodes, and has resulted in the loss of a fragment or fragments, the DRD self-healing function will auto-rebuild and auto-reconfigure the recovered fragments in another set of disks, maintaining the level of 3 parities.

The resiliency level, as mentioned earlier, can be set up to 6, boosting the HydraStor survival factor of 6 disks or nodes failure in the grid. See below of how the autonomous DRD recovery works:

Despite lacking the razzle dazzle of most Silicon Valley storage startups and upstarts, credit be given where credit is due. NEC HydraStor is indeed a strong show stopper.

However, in a market that is as fickle as storage, deduplication solutions such as HydraStor, EMC Data Domain, and HP StoreOnce, are being superceded by Copy Data Management technology, touted by Actifio. It was rumoured that EMC restructured their entire BURA (Backup Recovery Archive) division to DPAD (Data Protection and Availability Division) to go after the burgeoning copy data management market.

It would be good if NEC can take notice and turn their HydraStor “supertanker” towards the Copy Data Management market. That would be something special to savour.

P/S: NEC. Sorry about the title. I just couldn’t resist it 😉

The Prophet has arrived

By cfheoh | January 8, 2014 - 10:09 pm |January 8, 2014 Cloud, EMC, Falconstor, Filesystems, Object Storage, Openstack, Software-defined Datacenter, Storage Tiering, Unified Storage, Virtualization

Category Archives: Object Storage

Pure Electric!

The engineering of Elastifile

Oops, excuse me but your silo is showing

The reverse wars – DAS vs NAS vs SAN

Hail Hydra!

Recent Posts

Sponsored Ads

Google Adsense

Recent Comments

Google Adsense

Share this:

“No, we are not a storage company anymore. We are a data management company now.“

NetApp’s CIO Helps Drive Company’s Shift From Data Storage To Data Management

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Recent Posts

Sponsored Ads

Google Adsense

Recent Comments

Google Adsense

**“No, we are not a storage company anymore. We are a data management company now.“**