A big data reality check – what the hell is the use case?

Derek du Preez Profile picture for user ddpreez April 24, 2014
Research on the ever-popular topic of 'big data' provides a reality check - few businesses are doing much with it

© Paul Fleet - Fotolia.com
I stumbled across an interesting bit of research today carried out by The Information Difference, which focuses on the ever-popular topic of 'big data'. Now, as I've said before, I mostly try to steer clear of research papers because they are often just a commercial tool that self-serves the vendors that are sponsoring them – but this one caught my eye for a number of reasons. First of all, even though it is sponsored by helpIT and Teradata, the results don't really do them a lot of favours. Secondly, the sample seems decent; even though there are only just under 200 respondents, most of them came from companies with revenues in excess of $1 billion – proper enterprise examples that are also geographically pretty evenly distributed. Thirdly, the results give us some proper insight into what big data technologies companies are using,  how much they're spending and the challenges they are facing.

The results should serve as a bit of a reality check to those interested in the topic (including us media types that often write about the subject). From the results it seems that not only are businesses investing small amounts in big data rollouts, they are also struggling to pull together a business case for its use. For example, take a look at the respondents' differing stages of implementation:

Screen Shot 2014-04-25 at 10.47.42

Almost a third of all those surveyed (30%) currently have no plans to go live with a big data project in the near future, with only 27% stating that they are using tools to analyse large datasets. I would also be hesitant about the claim that 24% of respondents plan to roll something out over the next year, given that the number of live projects has only gone up 3 percent since last year – it's good that respondents are showing intention, but that's quite different to actually going live.

When focusing on the 27% live implementations – interestingly this splits out regionally as 6% in North America, 18% in Europe and 3% in the rest of the world. The authors of the research note that this contradicts previous assumptions about the US racing ahead in big data:

“This result flies in the face of conventional wisdom that suggests that North America is well ahead. Certainly, anecdotal evidence has been that the USA has been far ahead of Europe in terms of big data take-up, especially as things like web marketing analytics and social media were really the pioneers of Hadoop. 

“Given that 34% of respondents were drawn from North America, compared with 43% from Europe, it cannot reasonably be concluded that this difference can be wholly ascribed to the difference in sample sizes."

The nightmare business case

The report outlines how that 39% of respondents believe that big data projects will be implemented to improve the quality of their company's decision making, which was followed in second place by meeting customer needs (19%). It also found that it is IT that is driving the big data agenda, with 31% stating that the CIO is leading the project, followed by senior business managers (22%). However, things got really interesting when the researchers asked those with live projects what benefits they had realised to date:

Screen Shot 2014-04-25 at 10.50.15

As you can see, over half (51%) have no idea what benefits they are receiving from their big data projects. Not exactly a vote of confidence for those wanting to follow suite. Also, only 13% have identified new business opportunities and only 9% have been able to make more reliable decisions. This is very troubling indeed. If the early adopters aren't yet achieving any measurable benefits, the lack of exemplar use cases will be noticed and deter others from deploying technologies. This is seen in response to the question, what are the main roadblocks for big data initiatives?

The top three answers all relate to the business. Respondents stated that its very difficult to present a business case (33%), there is a lack of clear understanding of the business objectives (30%) and the management does not see it as imperative (27%). Also, take a look at the 'additional comments' that some of the respondents provided in their feedback to the researchers:

“Big Data seems to be buzz word only for us—needs to be more concrete.”  

“It's a solution looking for a problem in many cases.”

“Lots of moving parts with evolving use cases”

“The ability to separate the 'Big Noise' from 'Big Data' will be crucial for the success.”

“Too soon to know what the proper approach is.”

“We are still in the experimental stage, learning the technologies.”   

“We simply haven’t identified a use case yet—it isn't needed, and our current processes suffice.”

“Difficult to find a good business use case for it and then be able to demonstrate the value in a cost effective way.”

“As a business user, questions regarding data size are completely irrelevant.  I'm interested in capabilities—the tech choices are not of interest”

As I said before, these are comments from people that are involved with very large enterprises, which will have very large IT budgets. The

results seem to suggest that they are really struggling to not only get to grips with how the big data technologies work, but also the ROI. Companies largely still aren't sure how a big data project will give them better insight to help them make better decisions and to help them better serve their customers – key requirements if these projects are going to work. This lack of understanding is reflected in the big data spend, where it seems companies are not investing huge amounts in these projects. Only 8% of respondents indicated that they have spent over $5 million to date, while 42% have spent less than $1 million, 25% less than $100,000. Not exactly mind blowing.


  • A reality check is healthy. Vendors have huge marketing budgets and have been whipping up the hype around big data for the past couple of years because it could be a strong revenue stream for them going forward. If the results of this research are to be believed, it seems that users are still struggling to get to grips with big data and aren't really seeing very many benefits – yet. They also aren't spending a great deal.
  • Just because benefits aren't being realised at the moment, it doesn't mean that they won't be. To me, this just suggests that we are in the early stages of the maturity curve and companies are still figuring out the best use case for their business. As time goes on I'm sure this will change and bigger investments will be made. As we are constantly told, data is the new currency and those with the best understanding of their data will hold a competitive advantage.
  • From my experience speaking to companies that are investing in big data, they often have no idea what they are looking for and start small. They invest small amounts in the technologies to play with the tools alongside their traditional analytics platforms and start looking for those 'gold nuggets'. When they then start to find benefits in these small pilots, these are then used to create a use case and are put forward to argue for more investment. A sensible approach and one that reflects what's happening in reality.
A grey colored placeholder image