Category Archives: Data Security

Time to Conflate Storage with Data Services

By | | , , , , , , , , , , , , , , , , , ,
Leave a comment

Around the year 2016, I started to put together a better structure to explain storage infrastructure. I started using the word Data Services Platform before what it is today. And I formed a pictorial scaffold to depict what I wanted to share. This was what I made at that time.

Data Services Platform (circa 2016)- Copyright Heoh Chin Fah

One of the reasons I am bringing this up again is many of the end users and resellers still look at storage from the perspective of capacity, performance and price. And as if two plus two equals five, many storage pre-sales and architects reciprocate with the same type of responses that led to the deteriorated views of the storage technology infrastructure industry as a whole. This situation irks me. A lot.

Continue reading →

HODLing Decentralized Storage is not zero sum

By | | , , , , , , , , , ,
1 Comment

I have been dipping my toes into decentralized storage. I wrote about “Crossing the Chasm” last month where most early technologies have to experience to move into the mainstream adoption. I believe the same undertaking is going on for decentralized storage and the undercurrents are beginning to feel like a tidal wave. However, the clarion calls and the narratives around decentralized storage are beginning to sound the same after several months on researching the subject.

Salient points of decentralized storage

I have summarized a bunch of these arguments for decentralized storage. They are:

  • Democratization of cloud storage services separate from the hyperscaling behemoths of Web2
  • Inherent data security with default encryption, immutability and blockchain-ed. (most decentralized storage are blockchain-based. A few are not)
  • Data privacy with the security key for data decryption and authentication with the data owner(s)
  • No centralized control of data storage services, prices, market transparency and sovereignty
  • Green with more efficient energy consumption compared to Bitcoin
  • Data durability with data sharding creating no single point of failure and maintaining continuous data access services with geo content dispersal

Rocket fuel – The cryptos

Most early adoptions of a new technology require some sort of bliztscaling momentum to break free from the gravity of the old one. The cryptocurrencies pegged to many decentralized storage platforms are the rocket fuel to power the conversations and the narratives of the decentralized storage today. I probably counted over a hundred of these types of cryptocurrencies, with more jumping into the bandwagon as the gravy train moves ahead.

The table below is part of a TechTarget Search Storage article “7 Decentralized Storage Networks compared“. I found this article most enlightening.

7 Decentralized Storage Compared

Continue reading →

Rethinking data processing frameworks systems in real time

By | | , , , , , , , , , , , , , , , , , , , , ,
Leave a comment

“Row, row, row your boat, gently down the stream…”

Except the stream isn’t gentle at all in the data processing’s new context.

For many of us in the storage infrastructure and data management world, the well known framework is storing and retrieve data from a storage media. That media could be a disk-based storage array, a tape, or some cloud storage where the storage media is abstracted from the users and the applications. The model of post processing the data after the data has safely and persistently stored on that media is a well understood and a mature one. Users, applications and workloads (A&W) process this data in its resting phase, retrieve it, work on it, and write it back to the resting phase again.

There is another model of data processing that has been bubbling over the years and now reaching a boiling point. Still it has not reached its apex yet. This is processing the data in flight, while it is still flowing as it passes through processing engine. The nature of this kind of data is described in one 2018 conference I chanced upon a year ago.

letgo marketplace processing numbers in 2018

  • * NRT = near real time

From a storage technology infrastructure perspective, this kind of data processing piqued my curiosity immensely. And I have been studying this burgeoning new data processing model in my spare time, and where it fits, bringing the understanding back into the storage infrastructure and data management side.

Continue reading →

How well do you know your data and the storage platform that processes the data

By | | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Leave a comment

Last week was consumed by many conversations on this topic. I was quite jaded, really. Unfortunately many still take a very simplistic view of all the storage technology, or should I say over-marketing of the storage technology. So much so that the end users make incredible assumptions of the benefits of a storage array or software defined storage platform or even cloud storage. And too often caveats of turning on a feature and tuning a configuration to the max are discarded or neglected. Regards for good storage and data management best practices? What’s that?

I share some of my thoughts handling conversations like these and try to set the right expectations rather than overhype a feature or a function in the data storage services.

Complex data networks and the storage services that serve it

I/O Characteristics

Applications and workloads (A&W) read and write from the data storage services platforms. These could be local DAS (direct access storage), network storage arrays in SAN and NAS, and now objects, or from cloud storage services. Regardless of structured or unstructured data, different A&Ws have different behavioural I/O patterns in accessing data from storage. Therefore storage has to be configured at best to match these patterns, so that it can perform optimally for these A&Ws. Without going into deep details, here are a few to think about:

  • Random and Sequential patterns
  • Block sizes of these A&Ws ranging from typically 4K to 1024K.
  • Causal effects of synchronous and asynchronous I/Os to and from the storage

Continue reading →

At the mercy of the cloud deity

By | | , , , , , , , , , , , , , , , , , , ,
Leave a comment

Amazon Web Services (AWS) went down in the middle of last week. News of the outage were mentioned:

AWS Management Console unavailable error

Piling the misery

The AWS outage headlines attract the naysayers, the fickle armchair pundits, and the opportunists. Here are a few news articles that bring these folks to chastise the cloud giant.

Of course, I am one of these critics. I don’t deny that I am not. But I read this situation from a multicloud hyperbole of which I am not a fan. Too much multicloud whitewashing by vendors trying to pitch multicloud as a disaster recovery solution without understanding that this is easier said than done.

Continue reading →

Storage Elephant Compute Birds

By | | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Leave a comment

Data movement is expensive. Not just costs, but also latency and resources as well. Thus there were many narratives to move compute closer to where the data is stored because moving compute is definitely more economical than moving data. I borrowed the analogy of the 2 animals from some old NetApp® slides which depicted storage as the elephant, and compute as birds. It was the perfect analogy, because the storage is heavy and compute is light.

“Close up of a white Great Egret perching on top of an African Elephant aa Amboseli national park, Kenya”

Before the animals representation came about I used to use the term “Data locality, Data Mobility“, because of past work on storage technology in the Oil & Gas subsurface data management pipeline.

Take stock of your data movement

I had recent conversations with an end user who has been paying a lot of dollars keeping their “backup” and “archive” in AWS Glacier. The S3 storage is cheap enough to hold several petabytes of data for years, because the IT folks said that the data in AWS Glacier are for “backup” and “archive”. I put both words in quotes because they were termed as “backup” and “archive” because of their enterprise practice. However, the face of their business is changing. They are in manufacturing, oil and gas downstream, and the definitions of “backup” and “archive” data has changed.

For one, there is a strong demand for reusing the past data for various reasons and these datasets have to be recalled from their cloud storage. Secondly, their data movement activities still mimicked what they did in the past during their enterprise storage days. It was a classic lift-and-shift when they moved to the cloud, and not taking stock of  their data movements and the operations they ran on these datasets. Still ongoing, their monthly AWS cost a bomb.

Continue reading →

Sassy Cato

By | | , , , , , , , , , , , , ,
Leave a comment

I am not cybersecurity guy at all. Cybersecurity, to me, is a hodgepodge of many things. It is complex and it is confusing. But to every organization that has to deal with cloud SaaS (software-as-a-service) applications, mobile devices, work from home, and the proliferation of network connections from everywhere to the edge and back, strong cybersecurity without the burden of sluggish performance and without the complexity of stitching the cybersecurity point solutions would be a god send.

About 3 1/2 years ago, when I was an independent consultant, I was asked by a friend to help him (I was also looking for a gig) sell a product. It was Aryaka Networks, an SD-WAN solution. It was new to me, although I had some MPLS (multi protocol label switching) knowledge from some point in my career. But the experience with Aryaka at the people level was not too encouraging, with several people I was dealing with, switching positions or leaving Aryaka, including their CEO at the time, John Peters. After about 4 months or so, my friend lost confidence and decided to switch to Cato Networks.

Cato Networks opened up my eyes to what I believe cybersecurity should be. Simple, performant, and with many of the previous point requirements like firewall, VPN, zero trust networks, identity management, intrusion prevention, application gateways, threat detection and response, remote access, WAN acceleration and several more, all beautifully crafted into a single cloud-based service. There was an enlightenment moment for a greenhorn like me as I learned more about the Cato solution. That singularity of distributed global networking and cybersecurity blew me away.

Continue reading →

Control your Files. Control your Sovereignty.

By | | , , , , , , , , , , , , , , , , , ,
Leave a comment

Data residency, data sovereignty, data localization – the trio of data compliance and governance – have been on my mind a lot lately. I am seeing a disturbing trend. “Splinternet” has taken a hurried and hastened pace. We are now seeing many countries drawing up digital boundaries in the name of data privacy and data protection with sovereign laws and regulations. Besides, these digital demarcation along the lines with data definitions, digital “colonization” is a strong undercurrent as developing countries are accepting larger and more powerful foreign powers into their playpen.

Public cloud services transcend national borders. The breakneck speed in the adoption of public cloud services is causing anxieties and concerns with conservative governments everywhere. On the flip side of the coin, commerce has certainly flourished and bloomed as global wide collaborations bring new opportunities, new markets – all for capitalism and growth.

[ Note: While we are on this debacle, the voices of decentralization are getting louder as well, but that is a topic for another day ]

Where are your data files now?

Continue reading →

Right time for Andrew. The Filesystem that is.

By | | , , , , , , , , , , , , , , , , , , ,
Leave a comment

I couldn’t hold my excitement when I discovered Auristor® early last week. I stumbled upon this Computerweekly article “Want to side step Public Cloud? Auristor® offers global file storage.” Given the many news not exactly praising the public cloud storage vendors nowadays, the article’s title caught my attention. Immediately Andrew File System (AFS) was there. I was perplexed at first because I have never seen or heard a commercial version of AFS before. This news gave me goosebumps.

For the curious, I am sure many will ask who is this Andrew anyway? What is my relationship with this Andrew?

One time with Andrew

A bit of my history. I recalled quite vividly helping Intel in Penang, Malaysia to implement their globally distributed file caching mechanism with the NetApp® filer’s NFS. It was probably 2001 and I believed Intel wanted to share their engineering computing (EC) files between their US facilities and Intel Penang Design Center (PDC). As I worked along with the Intel folks, I found out that this distributed file caching technology was called Andrew File System (AFS).

Although I couldn’t really recalled how the project went, I remembered it being a bed of bugs at that time. But being the storage geek that I am, I obviously took some time to get to know Andrew the File System. 20 years have gone by, and I never really thought of AFS coming out as a commercial solution or even knew of it as one, until Auristor®,

Auristor Logo

Continue reading →

The Starbucks model for Storage-as-a-Service

By | | , , , , , , , , , , , , , , , , , , , , , , , , ,
Leave a comment

Starbucks™ is not a coffee shop. It purveys beyond coffee and tea, and food and puts together the yuppie beverages experience. The intention is to get the customers to stay as long as they can, and keep purchasing the Starbucks’ smorgasbord of high margin provisions in volume. Wifi, ambience, status, coffee or tea with your name on it (plenty of jokes and meme there), energetic baristas and servers, fancy coffee roasts and beans et. al. All part of the Starbucks™-as-a-Service pleasurable affair that intends to lock the customer in and have them keep coming back.

The Starbucks experience

Data is heavy and they know it

Unlike compute and network infrastructures, storage infrastructures holds data persistently and permanently. Data has to land on a piece of storage medium. Coupled that with the fact that data is heavy, forever growing and data has gravity, you have a perfect recipe for lock-in. All storage purveyors, whether they are on-premises data center enterprise storage or public cloud storage, and in between, there are many, many methods to keep the data chained to a storage technology or a storage service for a long time. The storage-as-a-service is like tying the cow to the stake and keeps on milking it. This business model is very sticky. This stickiness is also a lock-in mechanism.

Continue reading →