Can analytics make a company better? The answer isn't as easy as you think

NewVantage Partners (A Wavestone Company) just published their 11th Annual Survey of Chief Data Officers (CDOs) and Chief Data and Analytics Officers (CDAOs), the Data and Analytics Leaders Annual Executive Survey 2023, authored by Tom Davenport and Randy Bean.

Here are some of my comments about the results, not meant to criticize but to add some context from my own experience. Remember that these statistics are not from a random sample of businesses, but from an all-star group of 113 companies.

From Principal Challenge to Becoming Data-Driven, page 15:

Only 23.9% of organizations characterize themselves as data-driven, and only 20.6% say that they have developed a data culture within their organizations.

And, from Summary of 2023 executive survey participants by professional background and experience Page 9:

Cultural impediments continue to represent the greatest obstacle to corporate goals of achieving a data-driven organization and establishing a data culture within the firm. While there appears to be some improvement over time, an overwhelming majority –79.8% of data executives and business leaders – cite cultural impediments – people, business process, organizational alignment – as the primary barrier to data-driven organizational transformation.

There may be a better mix of opinions. It stands to reason that this group would point to culture as the major impediment, not technology or its implementation. But how difficult is the culture problem, and how skilled are the data people at recognizing it and moving it along? In my experience, the “data people'' have routinely blamed “culture,” meaning everyone else, for the failure of technology initiatives to achieve their goals.

In the '90s, when my firm was implementing DW and BI solutions, I referred to this as the Jordan River problem. We've wandered through the desert for decades, seemingly making progress, but when we get to the east bank of the Jordan, we can't get across. We implemented the technology and marshaled what, at the time, was considered a healthy arrangement of data, but the level of interest in using it productively needed to be there. When I  suggested we were ready to take it to the next stage, the response was, "Neil, aren't you the data guy? Maybe we should get McKinsey or BCG in here to figure it out."

Here is a radical idea. Is progress slow, or should we reexamine the whole data-driven digital transformation regime? It's hugely expensive and destabilizing in organizations. I’ll grant that data is an unsolved problem, but it isn’t the only one. The pressure originates from technology firms, consulting companies, and FOMO (fear of missing out). I wonder if the focus on technology, data modernization, and the "democratization of analytics" overlooks a more fundamental problem: it's still too difficult to apply. This pains me to say this. If progress in F500 companies is still far below what we projected, it's time to rethink the idea.

In an exchange with Davenport on LinkedIn, he said, "In short, while I agree we need to do something different, I don't think we need to give up on digital and data!"

Can analytics make a better company?

From Progress of Data and Analytics Aspiration on page 14:

The pattern of slow or negative progress is consistent across objectives. Managing data as a business asset – only 39.5%; have created a data-driven organization – a meager 23.9%; or established a data culture – a dismal 20.6%. Becoming data-driven is a long-term journey, and organizations continue to struggle to make substantive progress toward achieving this outcome. Less than half of organizations report that they are currently competing on analytics – just 40.8%

Davenport responded on LinkedIn: "41% of companies competing on analytics is pretty good -- I would estimate it was less than 1% when I wrote the article/book 15 years ago." According to Davenport, these companies were still relatively rare in 2007, with only about 1% of companies fitting his criteria. Keep in mind that 41% represents the proportion of the esteemed companies in the survey, while I suspect the 1% refers to all companies. However, analytics was becoming more widespread at the time, and it is likely that the number of companies using analytics in some capacity was higher than the number of "analytic competitors," as defined by Davenport.

I remember when Andrew White on the Gartner blog 2016, put it this way:

Having information (i.e. an analytic) that tells you that a train is running late is great. But what is more valuable is the act that follows the late notification, which helps you get to your meeting on time via other means. It is not the insight alone that creates value -  it's the resulting action (or inaction in the face of other actions) that delivers the real value. Thus the value of data is more likely impacted due to context – as in its use.

White recalled a client saying:

Analytics is getting in the way. Our business users equate the word to 'measuring things. I want them to think about re-imaging the business process, decision, and value realized. Analytics the word does not convey this scale of change.

The client who wanted people to re-imagine things instead of measuring them is spot on, but re-imagineers don't grow on trees. Whether analytics can make a better organization is still hanging in the air.

I have already gotten some DM's from colleagues to the effect: "OK, Neil, you shot off your big mouth that this whole approach is wrong. What's your suggestion?" As the late-great management consultant Leon Trotsky said, "Some questions are answered just by being asked." I haven't been a W-2 employee in an F500 company since 1985, though I spent the next 25 as a consultant in them. That is a very different experience. However, I could see how prevalent the resistance to "re-imagining" was.

The proposition of spending trillions of dollars on data infrastructure to make better decisions needs to be improved with some connective tissue, but it won’t be easy. I had a Topology professor who would only accept a proof up to two pages. I have some suggestions, I’ll save them for when I can compile them in two or fewer pages.

Re-imagining raises an issue about "data democratization." Democratization does not imply that every person in an organization will morph into an analytically inclined "user," or, for that matter, that senior executives will take up the mantle. Instead, the idea is that those who are inclined but have been frustrated with the technology and the complexity of the organizations will transition to analytically-oriented work. It implies a certain degree of non-uniformity. In other words, analysts will follow their ideas about things and follow different analytical paths all the time. I’m not convinced that proposals I’ve seen for virtualization, governance, security and collaboration can support that degree of non-uniformity.

Ethical and responsible AI

From the introduction, Page 7:

Companies continue to fall short in attention and commitment to data ethics policies and practices – Data ethics remains an issue of concern for data leaders as only 23.8% of companies report they are doing enough to ensure responsible and ethical use of data within their organizations and the industry.

On the State of Data Responsibility and Data Ethics, Page 21:

One might have hoped to see improvement in this area of responsible and ethical use of data and AI, and data trust. Sadly, this year's survey shows no meaningful improvement, as only 40.2% of data leaders reported that their organizations had made progress in establishing policies and practices for ethical data and AI use, a 9.0% decline from last year, and just 23.8% stated that the industry had undertaken actions to ensure responsible data and AI ethics. 76.2% say that their industry has still not done enough!

In a previous diginomica article, I reviewed a report, A Review of Why Are We Failing at the Ethics of AI? Anja Kaspersen and Wendell Wallach, senior fellows at Carnegie Council for Ethics in International Affairs take aim at the current discourse with three cutting points:

• Discussions of Ai Ethics and governance are too broad, and lack insight into the “subtleties and life cycles” of AI systems and their impacts. 
• “The talk about ethics is simply that: talk. 
• Discussions on AI and ethics are still "largely confined to the ivory tower."

On the last point, I would add organizations like UNESCO, that have been out in front with AI Ethics, which, given their history of internal corruption, is hypocritical.  The assumption behind lecturing people about ethics and promoting principles and declarations without remedies, is that people don’t know right from wrong. Most do. They just don’t know what to do about it. 

My take

This report is very useful and the authors have demonstrated their integrity over time (I take a dim view of some “research” conducted by vendors and some industry analysts). But I would argue (as I have above) that there can be some nuance to their interpretations. In fairness, I cherry-picked only a few.