Why is your digital transformation falling short? Start with leadership - and data

There was a time when IT had a stable of primarily incompatible applications. They spent a good deal of time building "interfaces," which failed when the applications upgraded.

In time, the legacy apps coalesced into a more comprehensive system, especially ERP. Still, it was a grueling, long-term, and expensive process for most companies that never delivered the single application to run everything. Extracts and interfaces and regular version upgrades of hardware, databases and software packages made life unpleasant.

The next wave, web services, promised seamless operation between applications, but the failure rate was too high. There were software failures, operator errors, hardware and environmental failures, and security violations. It was an excellent, elegant idea, but it didn't catch on. 

So, is it any wonder that when the Hadoop distributors offered an infinitely scalable and inexpensive alternative and eventually called it the "data lake, " it was irresistible? There was just one problem. It didn't work either. Putting hundreds or thousands of files into a Hadoop environment was like taking all of your papers and tossing them into a bin and feeling secure that you knew where they were. Except you didn't. The data lake concept survived, mostly without Hadoop, hosted by cloud services providers who offer inexpensive storage and computing of the applications, if desired. In truth, it is a workable solution, but not on its own - because it's more important to know what your data means than where it is.

History. I find it unbelievable that historical data is rarely mentioned, as if it is no different from current data. Suppose you are capturing weblogs for the past three or four years. Unless there have been no changes in the website (and maybe your problems now are more significant than transformation), the data will not match. Consider life insurance companies with customer data that may be thirty, forty, fifty years or more and have been ported to three or four or five different systems.

Now, digital transformation has become one of the most exciting terms in business today.
 
The Cambridge dictionary defines transformation as: "The process of changing completely the character or appearance of something to improve it." Unfortunately, we see many digital transformation plans that don't achieve much more than re-platforming applications and data, sliding newer technologies into the organization and hoping for ROI, principally to the cloud, to save money.

Applying a little logic, the major cloud providers are growing like weeds. Q12021 AWS revenue was $13.5 billion, up from 10.3 billion a year earlier. Microsoft was $15.1 billion, though that includes some other non-cloud revenue.  Google was $4.047 billion, an increase of 46% year-over-year. Although someone is paying those bills at $30 billion-plus and growing fast, the labyrinthian nature of cloud bills will continue to disguise what organizations are paying, and saving on it. 

That is not transformation.

Every digital transformation initiative will appeal to CEO leadership, which is freely given, then disappears as the project starts, only reappearing if the project is successful or at least part of it. Even in 2021, when leadership is from a new generation, their grasp of technology is insufficient. They see transformation as merely implementing new digital technologies. It's hard to understand when their role should ensure the organization shares one common digital vision. In general, we find CEOs are concerned with strategy, financials and dealing with the board. Visioning a reborn company through digital transformation below their pay grade is not only a skill they lack, it's not a skill the board hired them for. Support for transformation has to come from the next level down, the actual owners of the business units.

Unless the digital transformation changes the character or appearance of the organization, the initiative will languish or fail outright. But how do you do that? If all you achieve is automation, you are just automating old, inefficient and disjointed processes?

How do you change the character or appearance of your business, and to what? One approach is to free your talented people. Truly digital organizations encourage organization-wide digital skills and promoting the engagement of those skills. Look at the resources in your digital engagement. How many of them are purely IT? Are they the people to be thinking about reimagining the organization as a digital organization?

For many companies, digital business transformation is necessary for long-term survival in a competitive market. As time goes on, companies are becoming more data-endowed but not necessarily data-rich. So much is made of "data is the new oil" and monetizing data, but the truth is, data doesn't speak for itself. It acquires value through careful curation, governance and good models that put it in motion. So the idea of the data-driven organization is a fallacy. You can't drive an organization on data alone. 

So, suppose your digital transformation doesn't allocate a significant amount of time to managing data, applying functional semantic layers to it. In that case, permissions are strong enough to avoid scandal but loose enough for experimentation. It is a well-known fact that data scientists spend as much as 80% of their time finding and preparing data. Take care of that for them.

My take

To transform your company into a truly digital company is a monumental undertaking. To date, few companies have completed one. You can make technology decisions on your own or with guidance, but to use a hackneyed term, you need a change management plan if you're to transform. Perhaps you can conceive of your own, or you can use one of many available in the open market. Do you need a visionary to lead the project? Not necessarily, but you need a bunch of them in the project.