Want to double your pay? Become a Big Data expert.

Profile picture for user jmilne By Janine Milne November 10, 2014
There’s a rich seam of Big Data oil – but where are the expert drillers to dig it out? Survey reveals depth of Big Data skills deficit.

Want to double your pay? Then become a Big Data expert.

A shortage of skilled workers means UK Big Data professionals can command double the average wage, according to a SAS and Tech Partnership UK report. And the situation is set to get worse.

Demand for Big Data skills will intensify, increasing 160% between 2013 and 2020 in the UK, while the overall UK employment will grow just 6% over the same period.

Great news for Big Data specialists, a nightmare for organizations looking to beef up their Big Data exploration. In particular, finding data scientists - one of the key new roles to emerge with the rise of Big Data – is likely to tax both your recruitment team and your pockets.

The data scientist résumé requires expertise with maths, statistics, computer science, modeling and analysis. So far not so different from your average data or business analyst, but Geoffrey Taylor, SAS academic programme manager, points out that these are just the basics:

Increasingly there’s a requirement for people who are data scientists or specialists who understand what data is and how it flows though systems and organisations and can marshal it for the purpose of analysis and know what questions to ask of the data.

They need both “critical rigor and creative rigor”, observes Taylor, to be able to take a quantitative and qualitative view of data.

Others have described them as being part analyst, part artist or even Renaissance individuals. Taylor continues:

Having deep analytic skills is one thing, but it’s quite another to put that into a business context. You need to talk to IT or face the business and you need to be able to articulate in a credible way the challenges there are with the data.

They not only need to know how to dig out insights from data, but actually be able to identify the right problems or questions to ask that bring competitive advantage to the business. They must also be able to look at a problem from multiple angles rather than from within one area of expertise.

It’s this ability to see things from different perspectives that means data specialists won’t necessarily come out of traditional analytics or IT backgrounds. One senior data manager at a financial institution suggested that archaeologists would make great data scientists because of the way they look at the data in front of them. Taylor explains:

Archaeologists work with science and technology and if they find a tooth they talk about that took in context and think laterally what it could mean.

Screen Shot 2014-11-10 at 14.51.37

SAS UK and Ireland managing director, Mark Wilkinson, dubs Big Data as the “new oil” that will power the information economy. Big data analytics will refine this new oil so that valuable insights can be extracted.

He warns that Big Data is on the “cusp of going mainstream” as advances such as the Internet of Things gains traction, spewing out yet more data for organizations to sift through for nuggets of business insight.

In fact, Big Data is already mainstream in large organisations. A separate study by NewVantage Partners reveals that 67% of the Fortune 1000 senior business partners surveyed already have Big Data initiatives in production – a massive 32% hike from the same survey last year.

SAS is doing its part to increase Big Data skills levels, both with its own courses, but also by working with schools and universities. Taylor is working with business schools on a new breed of management science degrees, which use maths and statistical techniques to work with data.

He points out that:

typically these guys will disappear into the engine room of financial organizations or government departments.

No surprise then that the survey found that financial services were the key sector looking for big data skills, accounting for 20% of all positions in 2013.

There aren’t many of these talented data individuals to the pound, dollar or euro and it will take time for the investment in education to pay off. This is why, suggests Taylor, organizations should look to build an analytics team that together can dig out insight from the vast slag heap of big data.

But is the depth of the skills shortage really that deep? After all, analytics technology will improve, stripping out some of the complexity and therefore skills required. Taylor warns:

A little knowledge is a dangerous thing. So, yes, you can have point-and-click system, but if you don’t know what those statistics mean, then it’s dangerous. So we need a great deal more analytics talent.

The message is still don’t panic:

With each new wave of technology, there’s always a bubble but the industry is very creative when it comes to reskilling existing staff.

My take

The amount of information about Big Data skills emerging daily doesn’t require an analytics specialist to spell out the implications. There is a severe lack of skills, but this is exactly what you would expect from a new discipline. Organizations must identify the skills they need, but don’t look for them all in one person. Think team rather than individual and also extend the search beyond those with traditional data analytics capabilities.