This is the first part of the Hadoop blog series, starting from a quick recap from the Hadoop Summit 2015 in June. Next parts will be covering little more deeply technical and business aspects of the technological disruption caused by Hadoop and what changes organizations are facing on the road to be “Data-First” enterprises. Also scratching the topic “how, when, with whom?”, typical use cases and how Hadoop fits for example in the SAP landscape and architecture.
Hadoop – The Disruptive Technology of the Decade Part 1: Greetings from the Hadoop Summit 2015
In the beginning of June the 8th annual Hadoop Summit was held and co-hosted by Hortonworks and Yahoo!. The Hadoop Summit is the leading conference for the Apache Hadoop community. Where as the 1st Hadoop Summit 2008 hosted by Yahoo! had around 100 attendees, now more than 4000 people gathered to sunny and hot San Jose to catch the latest news, learn more about the Hadoop and hear user stories and cases. GE, Uber, TrueCar, Disney, eBay, PayPal, LinkedIn, Verizon, Rogers, Home Depot, Symantec, Schlumberger, Netflix, Yahoo!, just to name a few, who openly explained how they have not only solved their business issues with open source software around the Hadoop framework, but many times transformed their organization and business as well. Truly inspiring!
Some of the sessions by the sponsors or other technology vendors were too commercial for the nature of this Summit, but there were some hidden gems among as well. Otherwise many of the big sponsors weren’t that disappointing at all – but rather very surprising: there were many announcements towards contributing extensively to the open source community. Very nice side events were available as well, such as trainings, crash courses, partner events, industrial and strategy breakfast sessions and meet-ups. Two thumbs up for Hortonworks, Yahoo! and all the sponsors!
Having attended a lot of conferences around the world throughout my professional career, this time the enthusiasm and excitement were on a totally different level compared to any previous event. One could really feel that something special and disruptive is happening – right there, at that moment. And the forecasts backs up the same story as well: being $8 billion business at the moment (about evenly split between software, services and infrastructure), Hadoop is currently one of the quickest growing markets and even the most modest estimates for the year 2020 are more than $50 billion. Being a young market, e.g. Hortonworks has faced growth of 100+ new clients per quarter. Currently 40% of Fortune100 are either in production or experimenting Hadoop.
To summarize the Hadoop Summit 2015 in a couple of bullet points and few quotes:
- Hadoop is nearing a true enterprise maturity
- Hadoop is the disruptive technology of the decade
- If SQL is a gateway drug, you can’t live without Spark once you try it!
- Hadoop has shifted from techie2techie discussions to the board rooms
- Technology is now there, it needs to be about business from now on
- Hadoop is changing the way big enterprises are building their organizations
- Open source: Innovation at pace which proprietary software can never achieve
- Joining the ride essential for the old players as well to avoid being dinosaurs: announcements from Teradata, IBM, SAP to name a few – either to adapt Apache open source projects to their own platforms or to contribute to the community heavily
- Relevant for every business, in every industry
- Did I mention open source?
- And did I mention Spark?
A Few powerful quotes (might not be exactly word-by-word):
- SAP’s CTO Quentin Clark in his keynote: “Hadoop is the most important technological part of the digitalization”
- Forrester’s Principal Analyst Mike Gualteri in his keynote: “100 % of large (over $1 bil) enterprises adapts Hadoop by 2020”
- GE’s CIO Vince Campisi: “There is a clear technological gap in the market and Hadoop plays significant role filling that gap. Open standard approach is needed to keep up with the pace. Old technologies are not capable for billions of things to be connected.”
- Symantec’s Cloud Platform Engineering Leader David Lin: “”When you bring all data out of separate vertical silos, the collaboration process can then begin” & ” Haters to the left! Kill the fear! Just get it started and go!”
- TrueCar’s VP Technology John Williams: “Hadoop is enterprise ready! You can do really cool shit!”
Keynote videos and recorded sessions available online here and presentations here.
Last but not least “We are hiring…” board featuring Netflix, Tesla, Hortonworks, SAP, Accenture etc. There has never been such a big skill gap in the recruiting market, thus sometimes digital < analog. Up yours digitalization: