Storage Gaga

A little yellow elephant

By cfheoh | December 17, 2011 - 10:35 am |October 27, 2012 Analytics, Big Data, Data, Filesystems, Hadoop

Scheduling

By default Hadoop uses first-in, first-out (FIFO), and optional 5 scheduling priorities to schedule jobs from a work queue. In version 0.19 the job scheduler was refactored out of the JobTracker, and added the ability to use an alternate scheduler (such as the Fair scheduler or the Capacity scheduler).

Fair scheduler

The fair scheduler was developed by Facebook. The goal of the fair scheduler is to provide fast response times for small jobs and QoS (Quality of Service) for production jobs. The fair scheduler has three basic concepts.

Jobs are grouped into Pools.
Each pool is assigned a guaranteed minimum share.
Excess capacity is split between jobs.

By default jobs that are uncategorized go into a default pool. Pools have to specify the minimum number of map slots, reduce slots, and a limit on the number of running jobs.

Capacity scheduler

The capacity scheduler was developed by Yahoo. The capacity scheduler supports several features which are similar to the fair scheduler.

Jobs are submitted into queues.
Queues are allocated a fraction of the total resource capacity.
Free resources are allocated to queues beyond their total capacity.
Within a queue a job with a high level of priority will have access to the queue’s resources.

I took most the extract below from Wikipedia, and I don’t claim to be a knowledgeable person on Hadoop. All the credits go to Wikipedia editors to put Hadoop in layman terms.

Hadoop has certainly won the hearts of the new digital gold rush, Big Data and is slowly becoming a force to be reckoned with among data scientists. Hadoop implementations are powering new frontiers in processing and mining the ever growing data capacity, giving solution providers a simple programming methodology and data model to gain more insights into the vast seas of data and information.

Hadoop has many fans, and slowly becoming the data platform for large companies such as Yahoo!, Facebook, IBM, Amazon, Apple, eBay and many more. Facebook even claims to have the largest Hadoop clusters in the world, growing to 30PB in July of 2011.

This little yellow elephant is going places and one to watch out for.

Greenplum looking mighty sweet

By cfheoh | December 16, 2011 - 9:06 am |October 27, 2012 Big Data, EMC

1 Comment

Big data is Big Business these days. IDC predicts that between 2012 and 2020, the spending on big data solution will account for 80% of IT spending and growing at 18% per annum. EMC predicts that the big data is worth USD$70 billion! That’s a very huge market.

We generate data, and plenty of it. In the IDC Digital Universe Report for 2011 (sponsored by EMC), approximately 1.8 zettabytes of data will be created and replicated in 2011. How much is 1 zettabyte, you say? Look at the conversion below:

                    1 zettabyte = 1 billion terabytes

That’s right, folks. 1 billion terabytes!

And this “mountain” of data and information is a Goldmine of goldmines, and companies around the world are scrambling to tap on this treasure chest. According to Wikibon, big data has the following characteristics:

Very large, distributed aggregations of loosely structured data – often incomplete and inaccessible
Petabytes/exabytes of data
Millions/billions of people
Billions/trillions of records
Loosely-structured and often distributed data
Flat schemas with few complex interrelationships
Often involving time-stamped events
Often made up of incomplete data
Often including connections between data elements that must be probabilistically inferred

But what is relevant is not the definition of big data, but rather what you get from the mountain of information generated. The ability to “mine” the information from big data, now popularly known as Big Data Analytics, has sparked a new field within the data storage and data management industry. This is called Data Science. And companies and enterprises that are able to effectively use the new data from Big Data will win big in the next decade. Activities such as

Business decision making
Gain competitive advantage
Drive productivity growth in relevant industry segments
Understanding consumer and business behavioural patterns
Knowing buying decisions and business cycles
Yielding new innovation
Reveal customer insights
much, much more

will drive a whole new paradigm that shall be known as Data Science.

And EMC, having purchased Greenplum more than a year ago, has started their Data Computing Products Division immediately after the Greenplum acquisition. And in October of 2010, EMC announced their Greenplum Data Computing Appliance with some impressive numbers. Using 2 configurations of their appliance, noted below:

Below are 2 tables of the Greenplum performance benchmarks:

That’s what these big data appliance is able. The ability to load billions of either structured or unstructured files or objects in mere minutes is what drives the massive adoption of Big Data.

And a few days, EMC announced their Greenplum Unified Analytics Platform (UAP) which comprises of 3 Greenplum components:

A relational database for structured data
An enterprise Hadoop engine for the analysis and processing of unstructured data
Chorus 2.0, which is a social media collaboration tool for data scientists

The diagram below summarizes the UAP solution:

Greenplum is certainly ahead of the curve. Competitors like IBM Netezza, Teradata and Oracle Exalogic are racing to be ahead but Greenplum is one of the early adopters of a single platform for big data. Having a consolidation platform will not only reduce costs (integration of all big data components usually incurs high professional services’ fees) but will also reduce the barrier to entry to big data, thus further accelerating the adoption of big data.

Big Data is still very much at its infancy and EMC is pushing to establish its footprint in this space. EMC Education has already announce the general availability of courses related to big data last week and also the EMC Data Science Architect (EMC DSA) certification. Greenplum is enjoying the early sweetness of the Big Data game and there will be more to come. I am certainly looking forward to share more on this plum (pun intended ;-)) of the data storage and data management excitement.

“Ugly Yellow Box” bought by private equity firm

By cfheoh | December 11, 2011 - 7:05 am |October 27, 2012 Bluecoat, NetApp, Security

1 Comment

Security is BIG business, probably even bigger than storage and with more “sex” appeal and pazzazz! My friends are owners of 2 of the biggest security distributors in town, so I know. I am not much of a security guy, but I reason I write about Bluecoat is that this company has something close to my heart.

In the early 2000, NetApp used to have a separate division that is not storage. They have a product called NetCache, which is a web proxy solution. It was a pretty decent product and one of the competitors we frequently encounter on the field was an “ugly yellow box” called CacheFlow. Whenever we see an “ugly yellow box” in a rack, we will immediately know that it was a CacheFlow box. NetApp competed strongly with Cache Flow, partly because their CEO and founder, Brian NeSmith, as we NetAppians were told, was ex-NetApp. And there was some animosity between Brian and NetApp, up to the point that I recalled NetApp’s CEO then, Dan Warmenhoven, declaring that “NetApp will bury CacheFlow!“, or something of that nature. At that point, in the circa of 2001-2002, CacheFlow was indeed in a bit of a rut as well. They suffered heavy losses and was near bankcruptcy. A old news from Forbes confirmed Brian NeSmith’s near-bankcruptcy adventure.

CacheFlow survived the rut, changed their name to Bluecoat Systems, and changed their focus from Internet caching to security. Know why they are know as “Bluecoat”? They are the policemen of the Internet, and policemen are men in blue coats. I found an old article from Network World about their change. And they decided not to paint their boxes yellow anymore. 😉

Eventually, it was CacheFlow who triumphed over NetApp. And the irony was NetApp eventually sold the NetCache unit and its technology to BlueCoat in 2006. And hence, that my account of the history of Bluecoat.

Yesterday, Bluecoat was on the history books again, but for a better reason. A private equity firm, Thoma Bravo, has put in USD$1.3 billion to acquire Bluecoat. News here and here.

Have a happy Sunday 😀

Gartner 3Q2011 WW ECB Disk Storage Market

By cfheoh | December 10, 2011 - 9:54 am |October 27, 2012 Commvault, Dell, EMC, Gartner, HP, IBM, NetApp, Oracle, Storage Market Share

Cloud Computing and it’s not iCloud

By cfheoh | December 9, 2011 - 9:55 am |October 27, 2012 Amazon, Apple, Cloud, Steve Jobs

1 Comment

Steve Jobs was great with what he has done, but when it comes to Cloud Computing, Jeff Bezos of Amazon is the one. And I believe the Amazon Web Services (AWS) is bigger than Apple’s iCloud, in this present time and the future. Why do I say that knowing that the Apple fan boys could be using me as target practice? Because I believe what Amazon is doing is the future of Cloud Computing. Jeff Bezos is a true visionary.

One thing we have to note is that we play different roles when it comes to Cloud Computing. There are Cloud Service Providers (CSP) and there are enterprise subscribers. On a personal level, there are CSPs that cater for consumer-level type of services and there are subscribers of this kind as well. The diagram below shows the needs from an enterprise perspective, for both providers and subscribers.

Also we recognize Amazon from a less enterprise perspective, and they are probably better known for their engagement at the consumer level. But what Amazon is brewing could already be what Cloud Computing should be and I don’t think Apple iCloud is quite there yet.

Amazon Web Services cater for the enterprise and the IT crowd, providing both Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS) through its delectable offerings of the

Elastic Compute Cloud (EC2)
SimpleDB
Simple Storage Service (S3)
Elastic Block Store (EBS)
Elastic Beanstalk
CloudFormation
many more

And AWS has been operational and serving enterprise customers for 5-6 years now. Netflix, Zynga, Farmville are some of AWS customers. This is something Apple iCloud do not have, a Cloud Computing ecosystems for enterprise customers. Apple iCloud do not offer PaaS or IaaS. Perhaps that’s Apple vision not to get into the enterprise, but eventually the world evolve around businesses and businesses are adopting Cloud Computing. Many readers may disagree with what I say now in this paragraph but I will share with you later that even at the consumer level, Amazon is putting right moves in place, probably more so than Apple’s vision. (more about this later).

But the recent announcement of Kindle Fire, their USD$199 Android-based gadget, was to me, the final piece to Amazon’s Phase I jigsaw – the move to conquer the Cloud Computing space. I read somewhere that USD$199 Kindle Fire actually costs about USD$201.XX to manufacture. Apple’s iPad costs USD$499. So Amazon is making a loss for each gadget they sell. So what! It’s no big deal.

Let me share with you this table that will rattle your thinking a little bit. Remember this: Cloud Computing is defined as a “utility”. Cloud Computing is about services, content.

The table was taken from a recent Wired Magazine article. It featured the interview with Jeff Bezos. Go check out the interview. It’s very refreshing and humbling.

I hope the table is convincing you enough to say that the device or the gadget doesn’t matter. Yes, Apple and Amazon have different visions when it comes to Cloud Computing, but if you take some time to analyze the comparison, Amazon does not lock you into buying expensive (but very good) hardware, unlike Apple.

Take for instance the last point. Apple promotes downloaded media while Amazon uses streamed media. If you think about it, that what Cloud Computing should be because the services and the contents are utility. Amazon is providing services and content as a utility. Apple’s thinking is more old-school, still very much the PC-era type of mentality. You have to download the applications onto your gadget before you can use it.

Even the Amazon Silk browser concept is more revolutionary that Apple’s Safari. The Silk browser splits some of the processing in the Amazon Cloud, taking advantage of the power of the Amazon Cloud to do the processing for the user. Here’s a little video about Amazon Silk browser.

The Apple Safari is still very PC-centric, where most of the Web content has to be downloaded onto the browser to be viewed and processed. No doubt the Amazon Silk also download contents, but some of the processing such as read-ahead, applet-processing functions have been moved to Amazon Cloud. That’s changing our paradigm. That’s Cloud Computing. And iCloud does not have anything like that yet.

Someone once told me that Cloud is about economics. How incredibly true! It is about having the lowest costs to both providers and consumers. It’s about bringing a motherload of contents that can be delivered to you on the network. Amazon has tons of digital books, music, movies, TV and computing power to sell to you. And they are doing it at a responsible pace, with low margins. With low margins, the barrier of entry is lower, which in turn accelerates the Cloud Computing adoption. And Amazon is very good at that. Heck, they are selling their Kindle Fire at a loss.

Jeff Bezos has stressed that what they are doing is long term, much longer term than most. To me, Jeff Bezos is the better visionary of Cloud Computing. I am sorry but the reality is Steve Jobs wants high margins from the gadgets they sell to you. That is Apple’s vision for you.

Photo courtesy of Wired magazine.

What should be a Cloud Storage?

By cfheoh | December 8, 2011 - 1:43 pm |October 27, 2012 Analytics, Big Data, Filesystems, Object Storage

2 Comments

For us filesystem guys, NAS is the way to go. We are used to store files into network file systems via NFS and CIFS protocols and treating the NAS storage array like a refrigerator – taking stuff out and putting stuff back it. All that is fine and well as long as the data is what I would term as corporate data.

Corporate data is generated by employees, applications and users of the company and for a long time, the power of data creation lies in the hands of the enterprise. That is why storage solutions are designed to address the needs of the enterprise where the data is structured and well defined. How the data is stored; the data is formatted; and how is being accessed are the “boundary” of how the data is being used. Typically a database is used to “restrict” the data created so that the information can be retrieved and modified quickly. And of course, SAN guys will tell you to put these structured data of the data base into their SAN.

For the unstructured data in the enterprise, NAS file systems hold that responsibility. Files such as documents and presentations have a more loosely defined “boundaries”, and hence filesystems are a better natural fit for unstructured data. Filesystems are like a free-for-all container, and able to store and provide access to any files in the enterprise.

But today, as the Web 2.0 applications are already taking over the enterprise, the power of data creation does not necessary lie in the hands of the enterprise applications and users. In fact, it is estimated that the percentage of enterprise data now has exceeded 50% of the enterprise’s total data capacity. With the proliferation of personal devices such as tablets, Blackberries, smart phones, PDAs and so on, individual contributors are generating plenty of data. This situation has been made more acute with Web 2.0 applications, such as Facebook, blogs, social networking, Twitter, Google Search and so on.

Unfortunately, file systems in the NAS category still pretty much the traditional file systems, while the needs of a new type of file system could not be met by the traditional file systems. The paradigm is definitely shifting. The new unstructured data world needs a new storage concept. I would term this type of storage as “Cloud Storage” because it breaks down the traditional concepts of NAS.

So what basically defined a Cloud Storage? I already mentioned that the type of unstructured data has changed. And the new requirements for unstructured data type are:

The unstructured data type is capable of globally distributed.
There will be billions and billions of unstructured data objects created but each object, be it a Twitter tweet, or a uploaded mobile video, or even the clandestine data collected by CarrierIQ, can be accessed easily via a single namespace
The storage file system foundation for these new unstructured data type is easily provisioned and management. Look at Facebook. It is easy to setup, get going and the user (and probably the data administrator) can easily manage the user interface and the platform
For the service provider of Cloud Storage, the file system must be secure and support multi-tenancy and virtualization of storage resources
There should be some form of policy-driven content management. That is why development platforms such as Joomla!, Drupal, WordPress are slowing become enterprise driven to address these unstructured data types.
Highly searchable and have a high degree of search optimization. A Google search do have a strong degree of intelligence and relevance to the data being search as well as generating tons of by-product data that feeds the need to understand the consumers or the users better. Hail Big Data!

So when I compare traditional NAS storage solutions such as Netapp or EMC VNX or BlueArc, I ask the question of whether their NAS solutions has these capabilities to meet the requirements of these new unstructured data type.

Most of them, no matter how they package it, is still relying on files as the granular object of storage. And today, most files may have some form of metadata such as file name, owner, size etc, DO NOT, possess the capability of content-aware. Here’s an example when I want to show you:

The file properties (part of the file metadata) tell you about the file but little about the content of the file. Today, it requires more than that and the new unstructured data type should look more like this:

If you look at the diagram below, the object on the right (which is the new unstructured data type), display much more information than a typical file in a NAS file system. There additional information becomes the fodder to other applications such as search engines, RSS feeds, robots and spiders and of course, big data analytics.

Here’s another example of what I mean about these extended metadata, and a Cloud Storage storage array is required to work with these new set of parameters and a new set of requirement.

There’s a new unstructured data type in town. Traditional NAS systems may not have the right features to work with this new paradigm.

Don’t be white washed by the fancy talk of storage vendors in town. Learn the facts, and find out what is really a Cloud Storage.

It’s time to think differently. It’s time to think of what should be a Cloud Storage.

Storage must go on a diet

By cfheoh | December 5, 2011 - 1:38 pm |October 27, 2012 Backup, Data, EMC, NetApp, RAID, Reliability, Storage Optimization, Storage Tiering

1 Comment

Nowadays, the capacity of the hard disk drives (HDDs) are really big. 3TB is out and 4TB is in the horizon. What’s next?

For small-medium businesses in Malaysia, depending on their data requirements and applications, 3-10TB is pretty sufficient and with room to grow as well. Therefore, a 6TB requirement can be easily satisfied with 2 x 3TB HDDs.

If I were the customer, why would I buy a storage array, with the software licenses and other stuff that will not only increase my cost of equipment acquisition and data management, it will also increase the complexity of my IT infrastructure? I could just slot HDDs into my existing server, RAID it with RAID-0 (not a good idea but to save costs, most customers would do that) and I have a 6TB volume! It’s cheaper, easier to manage with Windows or Linux, and my system administrator doesn’t have to fuss about lack of storage experience.

And RAID isn’t really keeping up with the tremendous growth of HDD’s capacity as well. In fact, RAID is at risk. RAID (especially RAID 5/6) just cannot continue provide the LUN or volume reliability and data availability because it just takes too damn long to rebuild the volume after the failure of a disk.

Back in the days where HDDs were less than 500GB, RAID-5 would still hold up but after passing the 1TB mark, RAID-6 became more prevalent. But now, that 1TB has ballooned to 3TB and RAID-6 is on shaky ground. What’s next? RAID-7? ZFS has RAID-Z3, triple parity but come on, how many vendors have that? With triple parity or stronger RAID (is there one?), the price of the storage array is going to get too costly.

Experts have been speaking about parity-declustering, but that’s something that a few vendors have right now. Panasas, founded by one of forefathers of RAID, Garth Gibson, comes to mind. In fact, Garth Gibson and Mark Holland of Cargenie-Mellon University’s Parallel Data Lab (PDL) presented a paper about parity-declustering more than 10 years ago.

Let’s get back to our storage fatty. Yes, our storage is getting fat, obese, rotund or whatever you want to call it. And storage vendors have been pushing a concept in hope that storage administrators and customers can take advantage of it. It is called Storage Optimization or Storage Efficiency.

Here are a few ways you can consider to put your storage on a diet.

Compression
Thin Provisioning
Deduplication
Storage Tiering
Tapes and SSDs

To me, compression has not taken the storage world by storm. But then again, there aren’t many vendors that tout compression as a feature for storage optimization. Most of them rather prefer to push the darling of data reduction, data deduplication, as the main feature for save more space. Theoretically, data deduplication makes more sense when the data is inactive, and has high occurrence of duplicated data. That is why secondary storage such as backup deduplication targets like Data Domain, HP StoreOnce, Quantum DXi can publish 20:1 rates and over time, that rate can get even higher.

NetApp also has been pushing their A-SIS data deduplication on primary storage. Yes, it helps with the storage savings in primary but when the need for higher data transfer rates and time to access “manipulated” data (deduped or compressed), it is likely that compression is a better choice for primary, active data.

So who has compression? NetApp ONTAP 8.0.1 has compression now and IBM with its Storewize V7000 started as a compression device. Read about IBM Storewize in my blog here. Dell has Ocarina Networks, which was recently unleashed. I am a big fan of Ocarina Networks and I wrote about the technology in my previous blog. EMC, during the Celerra days of DART has compression but I don’t hear much about it in their VNX. Compression is there, believe me, embedded all the loads of EMC marketing.

Thin Provisioning is now a must-have and standard feature of all storage vendors. What is Thin Provisioning? The diagram below shows you:

In the past, storage systems aren’t so intelligent. You ask for 10TB, you are given 10TB and that 10TB is “deducted” from the storage capacity. That leads to wastage and storage inefficiencies. Today, Thin Provisioning will give you 10TB but storage capacity is consumed as it is being used. The capacity is not pre-allocated as in the past. Thin provisioning is a great diet pill for bloated storage projects.

Another up and coming feature is storage tiering. Storage tiering, when associated to storage optimization, should include hierarchical storage management (HSM) and tape-out as well. Storage optimization solutions should not offer only in the storage array itself. Storage tiering within the storage array is available with most vendors – IBM EasyTier, EMC FAST2, Dell Fluid Data Management and many others. But what about data being moved out of the storage array? What about reducing the capacity of the data online or near-line? Why not put them offline if there isn’t a need for it?

I term this as Active Archiving, something I learned while I was at EMC. Here’s a look at EMC’s style of Active Archiving:

Active Archiving promotes the concept of data archiving and is not unique only to EMC. Almost all storage vendors, either natively or with 3rd party vendors, can perform fairly efficient data archiving in one way or another. One of the software that I liked (and not unique!) is Quantum Stornext. Here’s a video of how Quantum Stornext helps reduce the fat of the storage.

With the single-copy sharing feature of Quantum Stornext to multiple disparate OSes, there are lesser duplicate files in storage as well.

Tapes have been getting a bad name in the past few years. It has been repositioned and repurposed as an archive medium rather than a backup medium. But tape is the greenest and most powerful storage diet pill around. And we should not be discount tapes because tapes are fighting back. Pretty soon you will be hearing about Linear Tape File System (LTFS). In a nutshell, Linear Tape File System (LTFS) allows you to use the tape almost as if it were a hard disk. You can drag and drop files from your server to the tape, see the list of saved files using a standard operating system directory (no backup software catalog needed), and use point and click to restore. How cool is that!

And Solid State Drives (SSDs) makes sense as well.

There are times that we need IOPS and using spinning drives, we have to set up many disk spindles to achieve the IOPS that we want. For example, using the diagram below from the godfather of storage, Greg Schulz,

The set of 16 spinning HDD drives on the left can only deliver 3,520 IOPS. The problem is, we have wasted a lot of disk space, as seen in the diagram below. This design, which most customer would be accustomed to, may look cheaper but in actual fact, is NOT.

If the price of a Fibre Channel HDD is RM2,000, the total of 16 would make up RM32,000.00. That is not inclusive of additional power and cooling and rack space and also the data management costs. Assuming the SSDs costs 5 times more than the Fibre Channel HDD. SSDs are capable of delivering very high IOPS. Here I am putting a modest 5,000 IOPS per SSDs. With just 2 SSDs (as the right design suggests), the total costs is only RM20,000. It has greater performance room to grow, and also savings in data management, power and cooling.

Folks, consider SSDs as part of your storage diet plan.

All these features are available, in whole or in part, and they are part of the storage technology offerings that is out there. With all these being said, are you doing something about it? Get off your lazy bum and start managing your storage and put your storage on a diet!!!

Hated GUI killing Ubuntu

By cfheoh | December 3, 2011 - 7:03 am |October 27, 2012 Uncategorized

3 Comments

OK, this is off-topic. Not my usual storage news but I thought I share this with you.

I am a Linux enthusiast. I play around with Linux – mostly Fedora and RedHat flavoured distros. For the past 2 years, one of the things I hated was the rise of Ubuntu. I don’t know why, but I just didn’t like the distro. Ubuntu, based on Debian, was the darling of the Linux desktop world. Perhaps I am a server guy but I just didn’t like Ubuntu. A few years ago, I won a Dell Latitude 2100 with Ubuntu pre-installed. I played around it with for a few days (hated it) and I decided to switch to Fedora 13 after that.

So, as Ubuntu’s star waned, I was piqued by the news. According to DistroWatch, which tracks popular Linux distros based on hits-per-day, Ubuntu is steadily on the decline. Here’s a look at the latest DistroWatch numbers of the top-10 Linux distros:

The decline is likely caused by Ubuntu Unity GUI, which replaced the likable GNOME/KDE interface in Ubuntu Natty Narwhal 11.04 version. The current version, Oneiric Ocelot 11.10, is taking a lot of hits of the wrong kind. It has dropped from the top spot and now down to #4.

Here’s a few screenshots of the Unity interface in Natty Narwhal.

I am pleasantly surprised that a GUI interface could cause so much harm to a Linux distro but judging by the number of haters out there, I guess the Unity GUI is killing Ubuntu’s popularity. Let’s see how Ubuntu will react in its version 12.04, Precise Pangolin.

The top distro is now Linux Mint, another Debian derivative. I have not tried Linux but I have been playing around with OpenSuSE 12.1. Not bad, buggy, but not bad.

I am still waiting to start my Fedora 16 download – 3.2GB baby over the Jaring SOMAport link. One day, but not today!

One-stop shop matters

By cfheoh | December 3, 2011 - 6:24 am |December 3, 2011 Storage Market Share

2 Comments

Would you buy fruits from dedicated fruit seller or would you go to a hypermarket to get your fruits? It depends on your preference but it is more likely that you would go to a hypermarket to do your shopping. You might need some accompanying stuff while you are at the hypermarket. There will be ideas stirring in your mind that you might need this or that while planning your fruit shopping.

The “ideas stirring in your mind” is what concepts like hypermarkets do. They mess around with your thinking and they play with your psychological side because we are human beings. We are driven by desire and convenience.

In storage, this whole psychological game comes into play as well in the customer’s purchasing habits. If the customer is purchasing storage from one vendor, he/she might as well get the rest of the data management solutions from the same vendor. The vendors would pitch easy, cost-effective, seamless, proven and other well-received words to woo the customer. And the key ingredient is INTEGRATION.

All solutions these days are complex, and integration of getting all components to work together is not easy. I have been working on a private cloud data appliance for almost 2 months now, and it’s not as seamless or as easy as it seems. According to the whitepaper, everything was rosy and dandy but when it comes down to ground zero, even the vendors themselves had a hard time doing the integration. And this drives up costs, resources and time.

That is why EMC has become a behemoth in the storage industry, being an A-Z one-stop shop of everything of data storage and management to every one. That is why IBM and HP are able to leverage their server business and their other solutions and services portfolio to entice the customers to buy their products. That is why Oracle wants to worn the whole bloody application stack in their Exadata, to sell more Oracle database licenses. Pure-play storage vendors like NetApp and HDS, who prefer to work on partnership could be feeling the heat of late.

In the latest IDC quarter worldwide disk storage system tracker (that’s a mouthful), NetApp is the prominent one being mentioned as “losing ground“. Here’s a look at a table, which compares past quarters results.

It is difficult to quantify integration costs, because there are many intangible, and unseen costs and impacts. To pacify customer’s fears, and increase their confidence in the total data storage and management solutions, marketing initiatives such as whitepapers, reference architectures, webcasts, social media, social business networking, demos, proof-of-concepts (POCs) and many more are tools of the trade that could tip a customer towards a vendor’s solution.

I believe NetApp could begin to realize that. And rumours are swirling in the industry for NetApp to acquire strong solutions such as Commvault and Quantum. It makes sense. NetApp is in need of a strong data protection solution in which it has a say in the vision and direction of the software. NetApp needs a strong data deduplication solution in which Quantum has in its DXi series. Symantec could be a acquisition target as well as the security and data management giant’s stock has stagnated in the stock market.

NetApp itself could be an acquisition target as well, with IBM, Cisco and HP the possible suitors. NetApp’s solutions are a great solution set for IBM, who really needs to do something about their staggered storage portfolio. HP might have chewed a mouthful with 3PAR but HP has been bad news for the last 2 quarters, no thanks to its on-and-off fiasco of ditching it PC business and other crazy stunts of HP-versus-Oracle and their ex-CEO, Leo Apotheker. Cisco could bet on NetApp too. Both companies have strong relationship together, but Cisco is drying up. They are becoming a laggard in the networking industry and companies like Juniper are hitting back … hard!

All these jousting and shuffling are creating the consolidation of the storage industry. The top six players – EMC, NetApp, IBM, HP, HDS and Dell – owns more than 80% of the total storage market share in terms of revenue. As the data storage and management world becomes more complex, and the ubiquity of cloud computing demands absolute uptime with no room for errors, the one-stop shop makes sense. One throat to choke … as they say.

Magic on storage players

By cfheoh | December 2, 2011 - 7:36 pm |December 2, 2011 Dell, EMC, Gartner, HP, IBM, NetApp, Storage Market Share

Storage Gaga

A little yellow elephant

Scheduling

Fair scheduler

Capacity scheduler

Greenplum looking mighty sweet

“Ugly Yellow Box” bought by private equity firm

Storage must go on a diet

Hated GUI killing Ubuntu

Recent Posts

Sponsored Ads

Google Adsense

Recent Comments

Google Adsense

Scheduling

Fair scheduler

Capacity scheduler

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Recent Posts

Sponsored Ads

Google Adsense

Recent Comments

Google Adsense