Practical Big Data series, Part 1
From hype to results
I probably don’t need to tell you how hyped up big data is these days. Chances are that you have already received more than your fair share of big data marketing. The hype phase of big data is also reflected in Gartner’s hype cycle graph, which shows that big data is just about to peak the cycle.
Thus, with all of the marketing noise out there, it is a tough challenge to understand what value there could be in big data for serious business. Yet, if you are a business leader, you need to decipher from the noise what is the right time and way to jump into the big data bandwagon. Too soon, and you pay a high price for the learning curve, too late and the competition has already wiped you off the table.
So how do we bust out of the hype? What is the right time to hop on? What is the recipe for success with big data?
This blog series provides a framework for and approach to big data that is tried-and-tested, down-to-earth, and provides tangible results. We call it Practical Big Data.
Let’s start first by exploring what’s wrong with big data as it presents itself to us right now:
What’s wrong with big data as we know it today?
Hint 1: Big data often starts out as a science rather than a business project
Many surveys like this and this indicate that around 50% of enterprise big data initiatives have failed. This demonstrates that big data is still very much in the early exploratory phases in most companies.
One of the commonly cited reasons for failure is the missing link to business. The missing link demonstrates itself as low understanding and ownership from the business side, inadequate resources, ill-defined goals, and lack of tangible end products delivered back to business.
As big data matures, you can no longer treat it as an R&D exercise. If you believe that big data has the potential to transform your industry landscape (and your company with it), the change should be managed from the board room instead of the lab. You would not leave an industry shaping merger for the R&D function to handle – so why would you leave big data either? Without proper C-level support, big data simply cannot deliver the enormous expectations that often are placed on it.
Hint 2: For businesses it’s not about the 3 V’s, it’s about Value
In the past, big data was defined as the 3 V’s: volume, variety, velocity. More recently a couple of more V’s were have been added i.e. variability and validity. Nevertheless, these are all technical measures of big data.
For businesses the only relevant measure of big data is VALUE. Without real tangible business benefit, few businesses will be able to support the big data initiative for long. Like any other major investment, a clear business case should be formulated for the big data project to work with.
Hint 3: Value is not in the data itself, but in how you put it into use
The trick, of course, is how to approach big data in a way that does provide value. Too often big data projects follow the formula “Big Data + Big Analysis + Big Insight = Big $$$”. This approach implicitly assumes that the data has intrinsic value. It usually does not.
More commonly, the value does not lie in the data itself, but in how fast you can put the data into practical business use. The forerunners of big data such as Facebook, eBay, Amazon, and Google have certainly realized this and use big data to predict what products, marketing messages, and prices they should show to their customers in real-time. These digital players have realized that also big data can have a best before date.
Hint 4: Some subsets of big data are more mature than others
The Gartner hype cycle also interestingly reveals is that real-time analytics is already making its road to productivity. In-memory analytics – i.e. analytics that enables real-time processing of terabytes of data – is already starting the upward slope. Furthermore, predictive analytics already seems to be delivering value to organizations that have managed to harness its power.
Real-time (in-memory) analytics and predictive analytics are both subcategories related to big data. While they may not in all their use cases strictly fulfill the original 3V’s of big data (volume, velocity, variety), in the big data end game you will most likely need both. As these technologies are already well ahead of the curve compared to the fuzzy “big data”, exploring these technologies first may give you an edge and route to faster results than starting your big data initiative e.g., with building a Hadoop cluster or setting up another NoSQL database.
Now that we know what is wrong with big data as we know it today, we are ready to turn our focus on how to do it right! Stay tuned for my next blog on Practical Big Data…!