[Disclosure: I was invited by GestaltIT as a delegate to their Storage Field Day 19 event from Jan 22-24, 2020 in the Silicon Valley USA. My expenses, travel, accommodation and conference fees were covered by GestaltIT, the organizer and I was not obligated to blog or promote the vendors’ technologies to be presented at this event. The content of this blog is of my own opinions and views]
This blog was not intended because it was not in my plans to write it. But a string of events happened in the Storage Field Day 19 week and I have the fodder to share my thoughts. Hadoop is indeed dead.
Warning: There are Lord of the Rings references in this blog. You might want to do some research. 😉
Storage metrics never happened
The fellowship of Arjan Timmerman, Keiran Shelden, Brian Gold (Pure Storage) and myself started at the office of Pure Storage in downtown Mountain View, much like Frodo Baggins, Samwise Gamgee, Peregrine Took and Meriadoc Brandybuck forging their journey vows at Rivendell. The podcast was supposed to be on the topic of storage metrics but was unanimously swung to talk about Hadoop under the stewardship of Mr. Stephen Foskett, our host of Tech Field Day. I saw Stephen as Elrond Half-elven, the Lord of Rivendell, moderating the podcast as he would have in the plans of decimating the One Ring in Mount Doom.
So there we were talking about Hadoop, or maybe Sauron, or both.
The photo of the Oliphaunt below seemed apt to describe the industry attacks on Hadoop.
Blow #1
Much was said about Hadoop’s implosion or the stay of the technology in the podcast. I am not going to steal the thunder of the podcast, which will be released next week. As the 4 of us argued, Stephen was at hand to herd the premise of “Is Hadoop dead?” and stirring the controversy that Hadoop was very much in the plans of many organizations which placed large enough investment in pachyderm’s technology.
At the debate battled, the argument went back and forth as the motion of for and against pitted both sides of the divide. In the end, as in the Battle of Helm’s Deep, Rohans vanquished the scourge of Saruman‘s armies of Orcs and Uruk-hais, giving Hadoop a fatal blow as the team of the for motion convinced the team who were against to switch allegiance that Hadoop was dying. And it was quick and swift.
Blow #2
Storage Field Day 19 Day 1 started and we reached NetApp mid morning. Their Active IQ® team shared their data driven customer experience support engine. As the architecture was exposed, we saw a few dead yellow elephant slides. Shades of Battle of Pelennor Fields appeared during the NetApp presentations and NetAppians, Gondorians alike, inflicting further fatality to Hadoop. When the core component of Active IQ® engine was revealed to be Prometheus and not Hadoop, its death became imminent. Sauron‘s demise is near. Hadoop’s future is nearing its final breath.
Ranking zombies counts
In our Storage Field Day Slack channel, Stephen shared his technology demise chart. “How dead is Hadoop?” measured in zombie counts. Personally, I would rate “OpenStack Dead” for Hadoop but pieces of the technology would remain useful in other ways, in other applications perhaps.
Nazgûl or the Black Riders, when they were vanquished, left many questions unanswered. Hadoop or pieces of it, may remain unanswered too through the timelines, a soundbite in the history books.
Cloudera switch and revival
A few weeks ago, Cloudera announced their new CEO, Rob Beardon to helm the company. He was previously running the show at previous rival, now partner Hortonworks. The decaying core pieces of MapReduce, HDFS and Yarn were removed from Cloudera Data Platform, replaced by modern, cloud-friendly substitutes of Apache Spark, Object Storage (probably Ozone) and Kubernetes.
The events bestowed upon Cloudera and its revival felt like Aragon has returned to Minas Tirith after the siege of the Mordor army, as the rightful heir to mankind to begin the Fourth Age, the dominion of men. It was truly the return of the king and the death of Hadoop.
The Death Certificate
Technologies come and go, and they come back again, like a pendulum. Bilbo Baggins wrote “There and Back Again” and the concept of Hadoop will return again in another shape and form. For now, I am laying it to rest in its forever peace.
[ Note: I apologize if this blog post was worded strongly, sounding like a death knell and may ruffle the feathers of naysayers and believers. I love Lord of the Rings although I never finished the book in full. I apologize for the mistakes too, if any. ]
Please read on:
- Shout out to Keiren Shelden’s blog on his experience in the On-premise IT Roundtable podcast.
- Shout out to Arjan Timmerman’s Day 1 report of Storage Field Day 19.
Pingback: Hadoop Is Truly Dead - LOTR Version - Tech Field Day
Pingback: Storage Field Day 19 – Wrap-up and Link-o-rama | PenguinPunk.net