Customer experience trumps technical excellence – Gartner BI reports


Contrasting Gartner reports indicate that user experience trumps technical excellence. What does this mean for buyers going forward?

The ‘consumerization of IT’ is a hot topic in 2013. Many of the incumbent vendors seem beleaguered as upstarts come in and nibble away at markets that have been immune from attack over many years. Nowhere is this more evident than in the burgeoning world of business intelligence (BI.) Here is some background.

  • Last week, Stuart Lauchlan caught up with Birst which, along with Tableau, QlikView, Tidemark and  many others is punching well above its weight.
  • At Workday Rising, Aneel Bhusri, co-CEO Workday hinted the company might make an acquisition in the operational analytics space. That aside, most analysts I speak with give Workday high marks for a user experience that sets the benchmark for enterprise applications.
  • Elsewhere, SAP is refurbishing its UI with Fiori.
  • Earlier in the year I delivered a keynote at Mastering SAP in Melbourne entitled, ‘I don’t give a F*%k About Your Code.’ (see above.) While the crowd wasn’t outright hostile, there were plenty of geeks willing to argue that the new upstarts cannot come close to SAP (in this case) in terms of breadth, depth and richness. That may well be so but customers are not convinced.

The message is clear – user experience trumps technical excellence. And customers are taking note. So where else is this evidenced?

MQ BI Feb 2013

In February 2013, Gartner issued a report entitled ‘Magic Quadrant for Business Intelligence and Analytics Platforms.’ I’m not usually a fan of Gartner’s MQs though I respect the fact they are widely used by customers as part of the decision making ‘pack.’ That report shows few surprises although it is interesting to see QlikView and Tableau appearing in the top right hand side, leaders quadrant. (See image above) So far so good you might think and one for IT to put in the bank during the selection process.

MQ_BI_2 - revised

But then contrast this with a later Gartner report entitled: Survey Analysis: Customers Rate Their BI Platform Vendors, 2013, issued July, 2013. (See image above.) I’ve highlighted the incumbents but more to the point, the difference in positioning between the report that talks to the technology is almost diametrically opposed to that which talks to the customer experience. That can’t be right can it?

What are these MQs telling us?

  1. Stating the bleeding obvious – technical excellence is not the same as user experience. That was OK when IT ruled the world of enterprise applications buying but doesn’t work so well when the balance of power shifts to line of business. While some of us might scoff at the notion of the CMO taking power away from the CIO, that trend is real.
  2. It is a truism that technical excellence doesn’t equate to user adoption. Anyone remember the old VHS v Betamax wars?  Several years ago, I saw an elegant solution in SAP Streamwork that got killed last year in favor of Jam. Why? Streamwork was a solution looking for a problem. Jam solves the problem.
  3. Technical excellence doesn’t equate to customer value. Value is only achieved when software becomes pervasive. No-one questions the ROI of email because we all use it. Can we say the same about business applications that are supposed to provide real help in a data driven world? Buyers are increasingly questioning the extent to which they are paying for shelfware that was originally purchased in anticipation of end user adoption. Vendors may not be aware but a good part of the reason that shelfware exists comes from the passive/aggressive user who, in BI, would rather default to a spreadsheet than wrestle with complex BI solutions.

But then not all MQs are created equal. Check this from Nucleus, developed in 2012.

Nucleus BI


Wow – this would appear to partially contradict Gartner’s view although the results shown are based upon a survey undertaken in 2012.  Regardless, Nucleus makes the points that: (PDF download)

Nucleus has seen a greater focus on quick and easy deployments focused on SaaS and hosted deployments where organizations do not need to invest in additional hardware or internal support resources…

… In a Moneyball world, organizations have re-evaluated and expanded the breadth of processes and departments that should make data-driven decisions. Organizations seek analytics tools that are robust enough to handle the scale and complexity of enterprise data, but are easy enough for line-of-business employees to use without providing significant training.


  • The world of enterprise applications is rapidly changing. Time horizons are collapsing and the clock is ticking.
  • Vendors will always argue the validity of positioning in analyst matrices, questioning methodologies and data quality. However, it is hard to argue against the combination of anecdotal evidence and commentary offered by analysts from all sides. More to the point, IT buyers who favor the incumbents will find it increasingly difficult to justify investments in solutions that are not delivering the end user experience needed to extract the kind of value business now demands.
  • Buyers will argue that whatever the deficiencies in upstart solutions, if they’re good enough and they’ll get broad use, then that is enough to justify a change. After all,technology always catches up and the upstarts are strongly incentivized to code needed features as quickly as possible.
  • More worryingly for incumbents, I wonder how Gartner’s view from its February report will be informed by its July report when it updates in 2014.
  • Is it game over for the big boys? Not at all. They have vast resources to throw at the problem. The question comes – will they see what the real problems are and respond in time?

Disclosure: SAP and Workday are premier partners at the time of writing

Image credits: Gartner and Nucleus Research, featured image: © ra2 studio –

    Comments are closed.

    1. vijayasankarv says:

      dahowlett some nuances 1. UX trumps tech in many cases but not all /most cases . UX typically trumps for last mile solutions, not upstream

    2. vijayasankarv says:

      dahowlett 2, Larger vendors also have big% of footprint in older tech, which skews MQ type aggregated reports.

    3. dahowlett says:

      vijayasankarv upstream? What do you mean please?

    4. dahowlett says:

      vijayasankarv Well…that should be telling vendors something eh?

    5. vijayasankarv says:

      dahowlett 3. UX paradigm for general purpose BI tools is still very old school even more newer solutions. That sucks

    6. dahowlett says:

      vijayasankarv Having used #EnSw BI solutions on and off since ’97 and tested all the big ones – this doesn’t surprise one bit.

    7. dahowlett says:

      vijayasankarv you mean newer are mimicking past? If so then au contraire my friend. Not what I’ve seen.

    8. vijayasankarv says:

      dahowlett traditional technology intensive solutions are necessary to get data with high quality before a better UX tool takes over

    9. vijayasankarv says:

      dahowlett yes indeed – that lesson needs to be learned by newer vendors too to avoid the mistakes older vendors did

    10. dahowlett says:

      vijayasankarv Sure – no-one disagrees, but it seems customers don’t see that in the same way they look at faucet and not pipes 😉

    11. vijayasankarv says:

      dahowlett newer software from older vendors might be really good . But they won’t show up significantly in MQ due to aggregation .

    12. dahowlett says:

      vijayasankarv Sure – but that’s not what users are seeing – ergo vendors not doing good job to deal with the obvious.

    13. vijayasankarv says:

      dahowlett no – new ones are only incrementally doing better , not radically. Very few use NLP or search for example

    14. dahowlett says:

      vijayasankarv Don’t lead me there…oh, you have…so in that case why are Tableau, QlikTech etc creaming incumbents in real deals? 😉

    15. vijayasankarv says:

      dahowlett there is a customer aspect to this too . Very few like upgrades . Over time – non SaaS BI will always face this issue

    16. dahowlett says:

      vijayasankarv But they don’t seem to mind switching 😉

    17. vijayasankarv says:

      dahowlett it is not always switching either – most I know are incremental purchases , not switches

    18. vijayasankarv says:

      dahowlett they may cream in categories where they started for some time for sure .But what is their real market share even with all growth?

    19. says:

      vijayasankarv1 Good counterpoint as always. My only caveat: buyers treasure these types of report and if taken for granted then they score one or the other on the buyer’s tick list of things that go towards making the decision. So your point about aggregation is well made – the question comes: what happens when the Gartner consultant turns up for more input? Will he caveat his own findings? Hardly. 
      However – I strongly believe that when customers are demanding those ‘last mile solutions’ and care less about the ‘pipes’ that get the water to the faucet then the incumbents need to watch out.

    20. dahowlett says:

      BirstBI Well – you are – aren’t you? Across all measures – that’s intersting to me

    21. BirstBI says:

      dahowlett Customer success: our 2 word mission statement. We believe it’s this attitude that’ll put us front & center for the future of #BI

    22. dahowlett says:

      BirstBI Good for you but also take into account vijayasankarv position.

    23. Joanie Mann says:

      You know those car commercials on TV, where the sales person is telling the customer about how great the warranty on the vehicle is?  Yeah – the one where the customer wants to know if they should buy a good car, or buy a car with a good warranty.  Makes you think, doesn’t it?