Customer experience trumps technical excellence – Gartner BI reports

The ‘consumerization of IT’ is a hot topic in 2013. Many of the incumbent vendors seem beleaguered as upstarts come in and nibble away at markets that have been immune from attack over many years. Nowhere is this more evident than in the burgeoning world of business intelligence (BI.) Here is some background.

  • Last week, Stuart Lauchlan caught up with Birst which, along with Tableau, QlikView, Tidemark and  many others is punching well above its weight.
  • At Workday Rising, Aneel Bhusri, co-CEO Workday hinted the company might make an acquisition in the operational analytics space. That aside, most analysts I speak with give Workday high marks for a user experience that sets the benchmark for enterprise applications.
  • Elsewhere, SAP is refurbishing its UI with Fiori.
  • Earlier in the year I delivered a keynote at Mastering SAP in Melbourne entitled, ‘I don’t give a F*%k About Your Code.’ (see above.) While the crowd wasn’t outright hostile, there were plenty of geeks willing to argue that the new upstarts cannot come close to SAP (in this case) in terms of breadth, depth and richness. That may well be so but customers are not convinced.

The message is clear – user experience trumps technical excellence. And customers are taking note. So where else is this evidenced?

MQ BI Feb 2013

In February 2013, Gartner issued a report entitled ‘Magic Quadrant for Business Intelligence and Analytics Platforms.’ I’m not usually a fan of Gartner’s MQs though I respect the fact they are widely used by customers as part of the decision making ‘pack.’ That report shows few surprises although it is interesting to see QlikView and Tableau appearing in the top right hand side, leaders quadrant. (See image above) So far so good you might think and one for IT to put in the bank during the selection process.

MQ_BI_2 - revised

But then contrast this with a later Gartner report entitled: Survey Analysis: Customers Rate Their BI Platform Vendors, 2013, issued July, 2013. (See image above.) I’ve highlighted the incumbents but more to the point, the difference in positioning between the report that talks to the technology is almost diametrically opposed to that which talks to the customer experience. That can’t be right can it?

What are these MQs telling us?

  1. Stating the bleeding obvious – technical excellence is not the same as user experience. That was OK when IT ruled the world of enterprise applications buying but doesn’t work so well when the balance of power shifts to line of business. While some of us might scoff at the notion of the CMO taking power away from the CIO, that trend is real.
  2. It is a truism that technical excellence doesn’t equate to user adoption. Anyone remember the old VHS v Betamax wars?  Several years ago, I saw an elegant solution in SAP Streamwork that got killed last year in favor of Jam. Why? Streamwork was a solution looking for a problem. Jam solves the problem.
  3. Technical excellence doesn’t equate to customer value. Value is only achieved when software becomes pervasive. No-one questions the ROI of email because we all use it. Can we say the same about business applications that are supposed to provide real help in a data driven world? Buyers are increasingly questioning the extent to which they are paying for shelfware that was originally purchased in anticipation of end user adoption. Vendors may not be aware but a good part of the reason that shelfware exists comes from the passive/aggressive user who, in BI, would rather default to a spreadsheet than wrestle with complex BI solutions.

But then not all MQs are created equal. Check this from Nucleus, developed in 2012.

Nucleus BI


Wow – this would appear to partially contradict Gartner’s view although the results shown are based upon a survey undertaken in 2012.  Regardless, Nucleus makes the points that: (PDF download)

Nucleus has seen a greater focus on quick and easy deployments focused on SaaS and hosted deployments where organizations do not need to invest in additional hardware or internal support resources…

… In a Moneyball world, organizations have re-evaluated and expanded the breadth of processes and departments that should make data-driven decisions. Organizations seek analytics tools that are robust enough to handle the scale and complexity of enterprise data, but are easy enough for line-of-business employees to use without providing significant training.


  • The world of enterprise applications is rapidly changing. Time horizons are collapsing and the clock is ticking.
  • Vendors will always argue the validity of positioning in analyst matrices, questioning methodologies and data quality. However, it is hard to argue against the combination of anecdotal evidence and commentary offered by analysts from all sides. More to the point, IT buyers who favor the incumbents will find it increasingly difficult to justify investments in solutions that are not delivering the end user experience needed to extract the kind of value business now demands.
  • Buyers will argue that whatever the deficiencies in upstart solutions, if they’re good enough and they’ll get broad use, then that is enough to justify a change. After all,technology always catches up and the upstarts are strongly incentivized to code needed features as quickly as possible.
  • More worryingly for incumbents, I wonder how Gartner’s view from its February report will be informed by its July report when it updates in 2014.
  • Is it game over for the big boys? Not at all. They have vast resources to throw at the problem. The question comes – will they see what the real problems are and respond in time?

Disclosure: SAP and Workday are premier partners at the time of writing

Image credits: Gartner and Nucleus Research, featured image: © ra2 studio –

Den Howlett
Following 20+ years in finance and IT related roles, Den Howlett became a freelance writer/analyst/commenter specialising in enterprise IT. I co-launched Information Week in the UK (1996-7) and was a contributor to numerous UK based trade magazines. Most recently, he was a long term columnist on ZDNet. Today, Howlett provides strategic product direction support to a variety of enterprise vendors along with delivering M&A due diligence services. The raw idea for diginomica came to him at a time when enterprise topics in media were being crushed by consumer stories. There had to be a better way. diginomica is that better way.
Den Howlett


Disruptor, enterprise applications drama critic, BS cleaner, buyer champion and foodie trying to suck less every day.
Den Howlett

Want a free weekly newsletter?

A weekly digest of diginomica's best

We never sell your details. Period

Leave a Reply

  • Joanie Mann says:

    You know those car commercials on TV, where the sales person is telling the customer about how great the warranty on the vehicle is?  Yeah – the one where the customer wants to know if they should buy a good car, or buy a car with a good warranty.  Makes you think, doesn’t it?

  • dahowlett says:

    BirstBI Good for you but also take into account vijayasankarv position.

  • BirstBI says:

    dahowlett Customer success: our 2 word mission statement. We believe it’s this attitude that’ll put us front & center for the future of #BI

  • dahowlett says:

    BirstBI Well – you are – aren’t you? Across all measures – that’s intersting to me

  • dahowlett says:

    vijayasankarv1 Good counterpoint as always. My only caveat: buyers treasure these types of report and if taken for granted then they score one or the other on the buyer’s tick list of things that go towards making the decision. So your point about aggregation is well made – the question comes: what happens when the Gartner consultant turns up for more input? Will he caveat his own findings? Hardly. 
    However – I strongly believe that when customers are demanding those ‘last mile solutions’ and care less about the ‘pipes’ that get the water to the faucet then the incumbents need to watch out.

  • vijayasankarv says:

    dahowlett they may cream in categories where they started for some time for sure .But what is their real market share even with all growth?

  • vijayasankarv says:

    dahowlett it is not always switching either – most I know are incremental purchases , not switches

  • dahowlett says:

    vijayasankarv But they don’t seem to mind switching ;)

  • vijayasankarv says:

    dahowlett there is a customer aspect to this too . Very few like upgrades . Over time – non SaaS BI will always face this issue

  • dahowlett says:

    vijayasankarv Don’t lead me there…oh, you have…so in that case why are Tableau, QlikTech etc creaming incumbents in real deals? ;)

  • vijayasankarv says:

    dahowlett no – new ones are only incrementally doing better , not radically. Very few use NLP or search for example

  • dahowlett says:

    vijayasankarv Sure – but that’s not what users are seeing – ergo vendors not doing good job to deal with the obvious.

  • vijayasankarv says:

    dahowlett newer software from older vendors might be really good . But they won’t show up significantly in MQ due to aggregation .

  • dahowlett says:

    vijayasankarv Sure – no-one disagrees, but it seems customers don’t see that in the same way they look at faucet and not pipes ;)

  • vijayasankarv says:

    dahowlett yes indeed – that lesson needs to be learned by newer vendors too to avoid the mistakes older vendors did

  • vijayasankarv says:

    dahowlett traditional technology intensive solutions are necessary to get data with high quality before a better UX tool takes over

  • dahowlett says:

    vijayasankarv you mean newer are mimicking past? If so then au contraire my friend. Not what I’ve seen.

  • dahowlett says:

    vijayasankarv Having used #EnSw BI solutions on and off since ’97 and tested all the big ones – this doesn’t surprise one bit.

  • vijayasankarv says:

    dahowlett 3. UX paradigm for general purpose BI tools is still very old school even more newer solutions. That sucks

  • dahowlett says:

    vijayasankarv Well…that should be telling vendors something eh?

  • dahowlett says:

    vijayasankarv upstream? What do you mean please?

  • vijayasankarv says:

    dahowlett 2, Larger vendors also have big% of footprint in older tech, which skews MQ type aggregated reports.

  • vijayasankarv says:

    dahowlett some nuances 1. UX trumps tech in many cases but not all /most cases . UX typically trumps for last mile solutions, not upstream