The Moneyball failure

We increasingly use data and analysis to inform business decisions often replacing business processes that relied on little more than gut instinct. There’s a long history of it.

Using one’s gut is far from perfect, but for almost forever, that’s all we had. It’s one of the reasons that middle-aged men with grey hair have historically populated the executive suite. In the last few years, a derivative idea, the HIPPO, has been trending. It simply means the “highest paid person’s opinion” and it reinforces the concept that big decisions have tended to be made on that person’s gut instinct.

The change to data and analytics has been instructive and very helpful to business. Who can forget the idea of Moneyball (book and film), in which a few statistically-gifted baseball nuts crafted World Series contending teams out of spare parts? But Moneyball has an implicit flaw not seen until quite recently. It depends, as all analysis does, on good, clean, and valid data and for that reason should only be applied in its pure form outside of baseball with care. Without care, you revert to the old IT maxim of GIGO or “garbage in, garbage out.”

We’ve just seen a painful example of GIGO play itself out in the US election. I am not here to pick sides. I am simply here to raise a concern for CRM and business in general over our new-found love for all things data and analytics oriented. For several election cycles we in the tech community have been impressed by the analytics that let politicos thread impossibly small paths through data to win elections. But while the analytics has been truly impressive, it also exposed our limited abilities in data collection.

Collection matters

The way we collect data is hugely important. In baseball, we’ve been collecting it for over a century and it is highly reliable—so much so, that it can feed analytics with almost no correction. It is completely objective and someday will be seen as a gold standard because no one ever has to ask how sure we are of the data or its integrity.

But asking people for their opinions in the real world could not be a more subjective exercise. Someone might answer your question and then act completely differently. This says a lot about how the national polls in the recent election could have been so wrong and it has meaning for CRM.

A purely quantitative collection of data about an inherently personal and subjective thing like voting is bound to produce heartburn in the people who construct the analysis. A much better approach to data collection resides in asking qualitative questions and only then quantifying them. So rather than cranking a lot of demographic data through an analytic model, what we really need to do is ask open-ended emotion-seeking questions about how people feel regarding issues.

I’ve written extensively about this in Solve for the Customer using the dimension of a customer community instead of a conventional data gathering survey. Vendors that use the open-ended, emotion-seeking question approach are making significant progress in understanding customers’ needs and aspirations and the information gleaned is good enough to drive spending decisions worth billions of dollars. A community approach is no less needed in politics today.

Let me give an anecdotal example. There was an installation at the Museum of Fine Arts in Boston over the weekend (it could still be there) and I think it was called Secrets. Participants were asked to type a secret into a computer screen in one part of the museum and it would be printed in another part of the place. I guess one of the points of the exercise is that secrets aren’t secret if you know how to ask the right question.

One of the more popular secrets that random participants divulged was that they were voting for or liked Donald Trump for President. That’s the kind of data gathering that gives a truer picture of reality and the kind that businesses need to collect. I am not saying this could have changed the election, but if you are a businessperson, understanding how to get true information from your customers—and how to test and verify it—is critical to your success. Without it you are not much better than a HIPPO dealing with GIGO.