It is a disaster. No matter what we do, the leaks and the cracks are appearing faster than we are fixing it. It is a global pandemic.
I am not talking about COVID-19, the pandemic that has affected our lives and livelihood for over a year. I am talking about the other pandemic – the compromise of security of data.
In the past 6 months, the data leaks, the security hacks, the ransomware scourge have been more devastating than ever. Here are a few big ones that happened on a global scale:
The multi-cloud for infrastructure-as-a-service (IaaS) era is not here (yet). That is what the technology marketers want you to think. The hype, the vapourware, the frenzy. It is what they do. The same goes to technology analysts where they describe vision and futures, and the high level constructs and strategies to get there. The hype of multi-cloud is often thought of running applications and infrastructure services seamlessly in several public clouds such as Amazon AWS, Microsoft® Azure and Google Cloud Platform, and linking it to on-premises data centers and private clouds. Hybrid is the new black.
Multi-Cloud, on-premises, public and hybrid clouds
And the aspiration of multi-cloud is the right one, when it is truly ready. Gartner® wrote a high level article titled “Why Organizations Choose a Multicloud Strategy“. To take advantage of each individual cloud’s strengths and resiliency in respective geographies make good business sense, but there are many other considerations that cannot be an afterthought. In this blog, we look at a few of them from a data storage perspective.
In the beginning there was …
For this storage dinosaur, data storage and compute have always coupled as one. In the mainframe DASD days. these 2 were together. Even with the rise of networking architectures and protocols, from IBM SNA, DECnet, Ethernet & TCP/IP, and Token Ring FC-SAN (sorry, this is just a joke), the SANs, the filers to the servers were close together, albeit with a network buffered layer.
A decade ago, when the public clouds started appearing, data storage and compute were mostly inseparable. There was demarcation of public clouds and private clouds. The notion of hybrid clouds meant public clouds and private clouds can intermix with on-premise computing and data storage but in almost all cases, this was confined to a single public cloud provider. Until these public cloud providers realized they were not able to entice the larger enterprises to move their IT out of their on-premises data centers to the cloud convincingly. So, these public cloud providers decided to reverse their strategy and peddled their cloud services back to on-prem. Today, Amazon AWS has Outposts; Microsoft® Azure has Arc; and Google Cloud Platform launched Anthos.
Something triggered my thoughts a few days ago. A few of us got together talking about climate change and a friend asked how green was the datacenter in IT. With cloud computing booming, I would say that green computing isn’t really the hottest thing at present. That in turn, leads us to one of the most voracious energy beasts in the datacenter, storage. Where is green storage in the equation?
What is green?
Over the past decade, several storage related technologies were touted as more energy efficient. These include
Tape – when tapes are offline, they do not consume power and do not require cooling
Virtualization – Virtualization reduces the number of servers and desktops, and of course storage too
MAID (Massive Array of Independent Disks) – the arrays spin down the HDDs if idle for a period of time
SSD (Solid State Drives) – Compared to HDDs, SSDs consume much less power, and overall reduce the cooling needs
Data Footprint Reduction – Deduplication, compression and other technologies to reduce copies of data
SMR (Shingled Magnetic Recording) Drives – Higher areal density means less drives but limited by physics.
The largest gorilla in storage technology
HDDs still dominate the market and they are the biggest producers of heat and vibration in a storage array, along with the redundant power supplies and fans. Until and unless SSDs dominate, we have to live with the fact that storage disk drives are not green. The statistics from Statistica below forecasts that in 2021, the shipment of SSDs will surpass HDDs.
Today the areal density of HDDs have increased. With SMR (shingled magnetic recording), the areal density jumped about 25% more than the 1Tb/inch (Terabit per inch) in the CMR (conventional magnetic recording) drives. The largest SMR in the market today is 16TB from Seagate with 18TB SMR in the horizon. That capacity is going to grow significantly when EAMR (energy assisted magnetic recording) – which counts heat assisted and microwave assisted – drives enter the market next year. The areal density will grow to 1.6Tb/inch with a roadmap to 4.0Tb/inch. Continue reading →
[Preamble: I have been invited by Dell Technologies as a delegate to their upcoming Dell Technologies World from Apr 30-May 2, 2018 in Las Vegas, USA. My expenses, travel and accommodation will be paid by Dell Technologies, the organizer and I was not obligated to blog or promote the technologies presented at this event. The content of this blog is of my own opinions and views]
This may seem a little strange. How does Industry 4.0 relate to Dell Technologies?
Recently, I was involved in an Industry 4.0 consortium called Data Industry 4.0 (di 4.0). The objective of the consortium is to combine the foundations of 5S (seiri, seiton, seiso, seiketsu, and shitsuke), QRQC (Quick Response Quality Control) and Kaizen methodologies with the 9 pillars of Industry 4.0 with a strong data insight focus.
Industry 4.0 has been the latest trend in new technologies in the manufacturing world. It is sweeping the manufacturing industry segment by storm, leading with the nine pillars of Industry 4.0:
[Preamble: I was a delegate of Storage Field Day 15 from Mar 7-9, 2018. My expenses, travel and accommodation were paid for by GestaltIT, the organizer and I was not obligated to blog or promote the technologies presented at this event. The content of this blog is of my own opinions and views]
Storage Field Day 15 was full of technology. There were a few avant garde companies in the line-up which I liked but unfortunately NetApp and IBM were the 2 companies that came in at the least interesting end of the spectrum.
IBM presented their SpectrumProtect Plus. The data protection space, especially backup isn’t exactly my forte when it comes to solution architecture but I know enough to get by. However, as IBM presented, there were some many questions racing through my mind. I was interrupting myself so much because almost everything presented wasn’t new to me. “Wait a minute … didn’t Company X already had this?” or “Company Y had this years ago” or “Isn’t this…??”
I was questioning myself to validate my understanding of the backup tech shared by the IBM SpectrumProtect Plus team. And they presented with such passion and gusto which made me wonder if I was wrong in the first place. Maybe my experience and knowledge in the backup software space weren’t good enough. But then the chatter in the SFD15 Slack channel started pouring in. Comments, unfortunately were mostly negative, and jibes became jokes. One comment, in particular, nailed it. “This is Veeam 0.2“, and then someone else downgraded to version 0.1.
The Register wrote a damning piece about NetApp a few days ago. I felt it was irresponsible because this is akin to kicking a man when he’s down. It is easy to do that. The writer is clearly missing the forest for the trees. He was targeting NetApp’s Clustered Data ONTAP (cDOT) and missing the entire philosophy of NetApp’s mission and vision in Data Fabric.
I have always been a strong believer that you must treat Data like water. Just like what Jeff Goldblum famously quoted in Jurassic Park, “Life finds a way“, data as it moves through its lifecycle, will find its way into the cloud and back.
And every storage vendor today has a cloud story to tell. It is exciting to listen to everyone sharing their cloud story. Cloud makes sense when it addresses different workloads such as the sharing of folders across multiple devices, backup and archiving data to the cloud, tiering to the cloud, and the different cloud service models of IaaS, PaaS, SaaS and XaaS.
I like the way Amazon is building their Cloud Computing services. Amazon Web Services (AWS) is certainly on track to become the most powerful Cloud Computing company in the world. In fact, AWS might already is. But they are certainly not resting on their laurels when they launched 2 new services in as many weeks – Amazon DynamoDB (last week) and Amazon Storage Gateway (this week).
I am particularly interested in the Amazon Storage Gateway, because it is addressing one of the biggest fears of Cloud Computing head-on. A lot of large corporations are still adamant to keep their data on-premise where it is private and secure. Many large corporations are still very skeptical about it even though Cloud Computing is changing the IT landscape in a massive way. The barrier to entry for large corporations is not something easy, but Amazon is adapting to get more IT divisions and departments to try out Cloud Computing in a less disruptive way.
The new service, is really about data storage and data backup for large corporations. This is important because large corporations have plenty of requirements for data storage and data to be backed up. And as we know, a large portion of the data stored does not need to be transactional or to be accessed frequently. This set of data is usually less frequently used, for archiving or regulatory compliance reasons, particular in the banking and healthcare industry.
In the data backup operations, the reason data is backed up is to provide a data recovery mechanism when a disaster strikes. Large corporations back up tons of data every day, weeks or month and this data only has value when there is a situation that requires data relevance, data immediacy or data recovery. Otherwise, it is just plenty of data taking up storage space, be it on disk or on tape.
Both data storage and data backup cost a lot of money, both CAPEX and OPEX. In CAPEX, you are constantly pressured to buy more storage to store the ever growing data. This leads to greater management and administration costs, both contributing heavily into OPEX costs. And I have not included the OPEX costs of floor space, power and cooling, people (training, salary, time and so on) typically adding up to 3-5x the operations costs relative to the capital investments. Such a model of IT operations related to storage cannot continue forever, and storage in the Cloud offers an alternative.
These 2 scenarios – data storage and data backup – are exactly the type of market AWS is targeting. In order to simplify and pacify large corporations, AWS introduced the Amazon Storage Gateway, that eases the large corporations to take some of their IT storage operations to the Cloud in the form of Amazon S3.
The video below shows the Amazon Storage Gateway:
The Amazon Storage Gateway is a piece of software “appliance” that is installed on-premise in the large corporation’s data center. It seamlessly integrates into the LAN and provides a SSL (Secure Socket Layer) connection to the Amazon S3. The data being transferred to the S3 is also encrypted with AES (Advanced Encryption Standard) 256-bit. Both SSL and AES-256 can give customers a sense of security and AWS claims that the implementation meets the data storage and data recovery standards used in the banking and healthcare industries.
The data storage and backup service regularly protects the customer’s data in snapshots, and giving the customer a rapid recovery platform should the customer experienced on-premise data corruption or data disruption. At the same time, the snapshot copies in the Amazon S3 can also be uploaded into Amazon EBS (Elastic Block Store) and testing or development environments can be evaluated and testing with Amazon EC2 (Elastic Compute Cloud). The simplicity of sharing and combining different Amazon services will no doubt, give customers a peace of mind, easing their adoption of Cloud Computing with AWS.
This new service starts with a 60-day free trial and moving on to a USD$125.00 (about Malaysian Ringgit $400.00) per gateway per month subscription fee. The data storage (inclusive of the backup service), costs only 14 cents per gigabyte per month. For 1TB of data, that is approximately MYR$450 per month. Therefore, minus the initial setup costs, that comes to a total of MYR$850 per month, slightly over MYR$10,000 per year.
At this point, I like to relate an experience I had a year ago when implementing a so-called private cloud for an oil-and-gas customers in KL. They were using the HP EVS (Electronic Vaulting Service) to an undisclosed HP data center hosting site in the Klang Valley. The HP EVS, which was an OEM of Asigra, was not an easy solution to implement but what was more perplexing was the fact that the customer had a poor understanding of what would be the objectives and their 5-year plan in keeping with the data protected.
When the first 3-4TB data storage and backup were almost used up, the customer asked for a quotation for an additional 1TB of the EVS solution. The subscription for 1TB was MYR$70,000 per year. That is 7x time more than the AWS MYR$10,000 per year cost! I have to salute the HP sales rep. It must have been a damn good convincing sell!
In the long run, the customer could be better off running their storage and backup on-premise with their HP EVA4400 and adding an additional of 1TB (and hiring another IT administrator) would have cost a whole lot less.
Amazon Web Services has already operating in Singapore for the past 2 years, and I am sure they are eyeing Malaysia as their regional market. Unless and until Malaysian companies offering Cloud Services know to use economies-of-scale to capitalize the Cloud Computing market, AWS is always going to be a big threat to CSP companies in Malaysia and a boon of any companies seeking cloud computing services anywhere in the world.
I urge customers in Malaysia to start questioning their so-called Cloud Service Providers if they can do what AWS is doing. I have low confidence of what the most local “cloud computing” companies can deliver right now. I hope they stop window dressing their service offerings and start giving real cloud computing services to customers. And for customers, you must continue to research and find out more which cloud services meet your business objectives. Don’t be flashed by the fancy jargons or technical idealism thrown at you. Always, always find out more because your business cost is at stake. Don’t be like the customer who paid MYR$70,000 for 1TB per year.
AWS is always innovating and the Amazon Storage Gateway is just another easy-to-adopt step in their quest for world domination.
Happy New Year! I am looking forward to the year of 2012.
Lately, I have been involved in Cloud Computing forums and I have been reading articles on Cloud Computing. I even took up a 5-day course on Cloud Computing in order to prepare myself for the inevitable. Yes, Cloud Computing is here to stay, but we joke about it, don’t we? I think the fun word of Cloud Computing is “cloudy“, which is indeed very true.
As I ingest more and more information about Cloud Computing, the definition of how different people has different perspective or opinion about Cloud Computing has never been “cloudier“. It is fuzzy, hazy, and confusing. And in the forums, many were saying that virtualization is Cloud Computing. What do you think?
I found that one definition of Cloud Computing very definitive, yet simple. This definition comes from the National Institute of Standards and Technology (NIST) of the US Department of Commerce. In its publication #800145, NIST defines Cloud Computing to have the following 5 essential characteristics (duplicated in verbatim):
On-demand self-service. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.
Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations).
Resource pooling. The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, and network bandwidth.
Rapid elasticity. Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.
Measured service. Cloud systems automatically control and optimize resource use by leveraging a metering capability1 at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.
The 5 essential characteristics are very important in determining whether Virtualization = Cloud Computing, and I know there are a lot of people out there that says that Virtualization equates Cloud Computing. Let’s see the table below:
Some readers might argue about the “YES” or “NO” in the above comparison, but I do not want to dwell on the matter. Yes, I believe that many of these things are doable in their own right but with different level of complexity and costs. The objective is to settle the arguments and confusions of Cloud Computing, accept some definitive terms and move on.
As you can see from the table above, Virtualization does not equate to Cloud Computing. We can say that Virtualization enables Cloud Computing to happen. It is the pre-cursor to Cloud Computing.
In Cloud Computing, there are different Service Models. NIST defines 3 different Service Models. They are:
Software as a Service (SaaS). The capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure2. The applications are accessible from various client devices through either a thin client interface, such as a web browser (e.g., web-based email), or a program interface. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
Platform as a Service (PaaS). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider.3 The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment.
Infrastructure as a Service (IaaS). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed applications; and possibly limited control of select networking components (e.g., host firewalls).
And NIST went on to define the Deployment Models of Cloud Computing as listed below:
Private cloud. The cloud infrastructure is provisioned for exclusive use by a single organization comprising multiple consumers (e.g., business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises.
Community cloud. The cloud infrastructure is provisioned for exclusive use by a specific community of consumers from organizations that have shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be owned, managed, and operated by one or more of the organizations in the community, a third party, or some combination of them, and it may exist on or off premises.
Public cloud. The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider.
Hybrid cloud. The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).
There! Cloud Computing by the definition of NIST. It is simple, easily understood and most importantly, it give us the context of what we are looking for in the sea of confusion. Here’s the link to NIST’s PDF.
We can argue till the cows come home but it is best to stick to a simple definition of Cloud Computing and focus on other more important aspects of the cloud.
I hope to share more of my Cloud Computing experience with you and storage will have a big part to play in it.
Steve Jobs was great with what he has done, but when it comes to Cloud Computing, Jeff Bezos of Amazon is the one. And I believe the Amazon Web Services (AWS) is bigger than Apple’s iCloud, in this present time and the future. Why do I say that knowing that the Apple fan boys could be using me as target practice? Because I believe what Amazon is doing is the future of Cloud Computing. Jeff Bezos is a true visionary.
One thing we have to note is that we play different roles when it comes to Cloud Computing. There are Cloud Service Providers (CSP) and there are enterprise subscribers. On a personal level, there are CSPs that cater for consumer-level type of services and there are subscribers of this kind as well. The diagram below shows the needs from an enterprise perspective, for both providers and subscribers.
Also we recognize Amazon from a less enterprise perspective, and they are probably better known for their engagement at the consumer level. But what Amazon is brewing could already be what Cloud Computing should be and I don’t think Apple iCloud is quite there yet.
Amazon Web Services cater for the enterprise and the IT crowd, providing both Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS) through its delectable offerings of the
Elastic Compute Cloud (EC2)
SimpleDB
Simple Storage Service (S3)
Elastic Block Store (EBS)
Elastic Beanstalk
CloudFormation
many more
And AWS has been operational and serving enterprise customers for 5-6 years now. Netflix, Zynga, Farmville are some of AWS customers. This is something Apple iCloud do not have, a Cloud Computing ecosystems for enterprise customers. Apple iCloud do not offer PaaS or IaaS. Perhaps that’s Apple vision not to get into the enterprise, but eventually the world evolve around businesses and businesses are adopting Cloud Computing. Many readers may disagree with what I say now in this paragraph but I will share with you later that even at the consumer level, Amazon is putting right moves in place, probably more so than Apple’s vision. (more about this later).
But the recent announcement of Kindle Fire, their USD$199 Android-based gadget, was to me, the final piece to Amazon’s Phase I jigsaw – the move to conquer the Cloud Computing space. I read somewhere that USD$199 Kindle Fire actually costs about USD$201.XX to manufacture. Apple’s iPad costs USD$499. So Amazon is making a loss for each gadget they sell. So what! It’s no big deal.
Let me share with you this table that will rattle your thinking a little bit. Remember this: Cloud Computing is defined as a “utility”. Cloud Computing is about services, content.
The table was taken from a recent Wired Magazine article. It featured the interview with Jeff Bezos. Go check out the interview. It’s very refreshing and humbling.
I hope the table is convincing you enough to say that the device or the gadget doesn’t matter. Yes, Apple and Amazon have different visions when it comes to Cloud Computing, but if you take some time to analyze the comparison, Amazon does not lock you into buying expensive (but very good) hardware, unlike Apple.
Take for instance the last point. Apple promotes downloaded media while Amazon uses streamed media. If you think about it, that what Cloud Computing should be because the services and the contents are utility. Amazon is providing services and content as a utility. Apple’s thinking is more old-school, still very much the PC-era type of mentality. You have to download the applications onto your gadget before you can use it.
Even the Amazon Silk browser concept is more revolutionary that Apple’s Safari. The Silk browser splits some of the processing in the Amazon Cloud, taking advantage of the power of the Amazon Cloud to do the processing for the user. Here’s a little video about Amazon Silk browser.
The Apple Safari is still very PC-centric, where most of the Web content has to be downloaded onto the browser to be viewed and processed. No doubt the Amazon Silk also download contents, but some of the processing such as read-ahead, applet-processing functions have been moved to Amazon Cloud. That’s changing our paradigm. That’s Cloud Computing. And iCloud does not have anything like that yet.
Someone once told me that Cloud is about economics. How incredibly true! It is about having the lowest costs to both providers and consumers. It’s about bringing a motherload of contents that can be delivered to you on the network. Amazon has tons of digital books, music, movies, TV and computing power to sell to you. And they are doing it at a responsible pace, with low margins. With low margins, the barrier of entry is lower, which in turn accelerates the Cloud Computing adoption. And Amazon is very good at that. Heck, they are selling their Kindle Fire at a loss.
Jeff Bezos has stressed that what they are doing is long term, much longer term than most. To me, Jeff Bezos is the better visionary of Cloud Computing. I am sorry but the reality is Steve Jobs wants high margins from the gadgets they sell to you. That is Apple’s vision for you.
ex·tra·or·di·naire – Outstanding or remarkable in a particular capacity
I was plucking the Internet after dinner while I am holidaying right now in Port Dickson. And at about this time, the news from my subscriptions will arrive, perfectly timed as my food is digesting.
And in the news – “IDC Says Cloud Adoption Fuels Storage Sales”. You think?
We are generating so much data in this present moment, that IDC is already saying that we are doubling our data every 2 years. That’s massive and a big part of it is being fueled our adoption to Cloud. It doesn’t matter if it is a public, private or hybrid cloud because the way we use IT has changed forever. It’s all too clear.
Amazon has a massive repository of contents; Google has been gobbling tons of data and statistics since its inception; Apple has made IT more human; and Facebook has changed the way we communicate. FastCompany magazine called Amazon, Apple, Google and Facebook the Big 4 and they will converge sooner or later into what the tornado chasers call a Perfect Storm. Every single effort that these 4 companies are doing now will inevitably meet at one point, where content, communication, computing, data, statistics all become the elements of the Perfect Storm. And the outcome of this has never been more clearer. As FastCompany quoted:
“All of our activity on these devices produces a wealth of data, which leads to the third big idea underpinning their vision. Data is like mother’s milk for Amazon, Apple, Facebook, and Google. Data not only fuels new and better advertising systems (which Google and Facebook depend on) but better insights into what you’d like to buy next (which Amazon and Apple want to know). Data also powers new inventions: Google’s voice-recognition system, its traffic maps, and its spell-checker are all based on large-scale, anonymous customer tracking. These three ideas feed one another in a continuous (and often virtuous) loop. Post-PC devices are intimately connected to individual users. Think of this: You have a family desktop computer, but you probably don’t have a family Kindle. E-books are tied to a single Amazon account and can be read by one person at a time. The same for phones and apps. For the Fab Four, this is a beautiful thing because it means that everything done on your phone, tablet, or e-reader can be associated with you. Your likes, dislikes, and preferences feed new products and creative ways to market them to you. Collectively, the Fab Four have all registered credit-card info on a vast cross-section of Americans. They collect payments (Apple through iTunes, Google with Checkout, Amazon with Amazon Payments, Facebook with in-house credits). Both Google and Amazon recently launched Groupon-like daily-deals services, and Facebook is pursuing deals through its check-in service (after publicly retreating from its own offers product).”
Cloud is changing the way we work, we play, we live and data is now the currency of humans in the developed and developing worlds. And that is good news for us storage professionals, because all the data has to eventually end up in a storage somewhere, somehow.
That is why there is a strong demand for storage networking professionals. Not just any storage professionals but the ones that have the right attitude to keep developing themselves, enhancing their skillset, knowledge and experience. The ones that can forsee that the future will worship them and label them as deities of the Cloud era.
So why are you guys take advantage of this? Well, don’t just sit there and be ordinary. Be a storage extraordinaire now! And for those guys who want to settle of being ordinary … too bad! I said this before – You could lose your job.
Photo courtesy of Wired magazine.