The pandemic has shown the positive role that government can play in using digital tools and data to transform the way organizations deliver value to citizens and businesses. That’s the view of Barbara Ubaldi, Digital Government and Open Data Lead at the OECD.
But there are deep problems with public sector digital transformation, she observed at this week’s Westminster eForum on the subject.
Technology is an enabler, not a goal. We shiver [sic] when we see that the focus is about the use of AI and not the value that using AI can bring.
It's vital to shift to an understanding of the importance of having a user-driven approach. Rather than interpreting the needs of the users, try to provide a space for users to express their own needs, so the solution is a response to their requests.
In short, government needs to listen more and talk less.
Unless government grows stronger around the following dimensions – being digital by design, being data driven, operating as a platform, and being open by default – it is not possible for digital transformation to grow more mature, to be user driven and proactive.
So, for us, when we talk about these things, we see data governance as a foundational necessity for value and trust. Reinforcing public trust means providing services to citizens and businesses that bring real value to them.
Here today, gone tomorrow
So, why is digital transformation so difficult? What are the barriers to success and the common mistakes in rolling out public-sector digital programmes? In short, why do so many government IT projects fail?
Dr Sam De Silva is Partner at international law firm CMS. He told the eForum:
The first point is, there's a disconnect between the organization's strategic priorities and the outcomes of a project. Projects are often thought about in terms of improving customer service, or making cost savings, but they're not linked to the business case of the project. And if they are linked to it, there's very little feedback or evaluation once the project is up and running.
The second point is digital transformation projects are often led by commercial and operational people. There's often no senior management or minister ownership and leadership. Often the minister or senior manager will say, ‘We need to get this project done’, but that's pretty much their only involvement in it. They sign the contract, but there's very little involvement during implementation or development.
One reason for this is that ministers are often (to quote the late political commentator Robin Day) here today and gone tomorrow. Even if the same party remains in power for a decade or more, as is currently the case in the UK, Cabinet churn and reshuffles make politics into a game of musical chairs. In that environment, continuity is almost impossible, especially when the constant machinery of government, the Civil Service, is itself a target for change.
De Silva continued:
The third point is these projects need engagement with stakeholders. But who are the stakeholders? Who are the people who really benefit from them?
My previous report suggested that, often, it is the internal job market in Whitehall that benefits, plus the revenue streams of large consultancies.
De Silva added:
The fourth point is project management and risk management are disciplines in themselves. They demand skills and expertise, but often the people who are managing these projects don't have those skills, qualifications, or experience.
Fifth is a lack of understanding about the supplier within the customer organization. Often proposals are evaluated by the initial price, rather than the long-term value for money. And sometimes these projects are undertaken in a big-bang, return-key approach, rather than being split into discrete modules or things can be managed appropriately.
And finally, why these projects fail is often the contract is signed and put in the bottom drawer and no one ever looks at again.
That’s quite a list and certainly describes many high-profile, big-bang IT projects seen in the UK over the past 25 years – the NHS National Programme for IT, for example. We can throw into this toxic mix the fact that policy change can happen within governments as well as between administrations. Take former Cabinet Office Minister Francis Maude’s personal drive to tackle the ‘oligopoly’ of big IT suppliers, only for them to drift back in after his replacement. Meanwhile, the country has had three Prime Ministers in the space of eleven years.
Fit for purpose contracts
These are the challenges. But what are the solutions? De Silva set out his own thoughts on the subject:
Key success factors include having a strong client/supplier relationship. In my experience, I've seen both ends of the spectrum. At one end, clients – local authorities or government departments – try to transfer all the risk to the supplier. But at the other extreme, the customer doesn’t think they need a robust contract at all. They believe everything can be done on a one-page document. They think the supplier is friendly, so if problems occur, they believe they can just sort it out.
A middle ground needs to be found where there's a balanced contract. This leads to my next point about contractual arrangements. I’ve seen clients present contracts that are just not fit for purpose. A contract that is good for buying furniture or stationery is not fit for digital transformation. I've also seen customers present a software licence agreement to procure software as a service. So more robust contractual arrangements are needed.
Now in terms of the development methodology, a waterfall approach is often criticized because people think it's not flexible enough. They think what’s needed is an agile or rapid application development methodology. But sometimes the traditional approach to development is still relevant, and sometimes it’s not. You need to look at the circumstances of the project and decide which methodologies are appropriate. This is a core part of contractual management and of digital transformation.
Then, what are the requirements that the software, solution, or service provider needs to deliver? These are often long-term projects, maybe five or 10 years, so change will happen. This means that a robust change-management process needs to be included, to ensure that the customer is not exposed to additional costs.
One organization that understands these issues only too well is the National Audit Office (NAO). Yvonne Gallagher is Digital Director at the National Audit Office (NAO). She said:
Government has repeatedly set out a vision for radical digital change, but really the pace has been so slow that we must conclude it has underestimated how easy it is to achieve that change in the real world.
Government isn't a greenfield site or a start-up. It's full of complexity and messy legacy systems, processes, and data. So, anything new must fit in with that. It’s relatively easy to start up something new with a clean slate but transforming what's already there is hard. It’s not just about failures of projects or programme management. A lot of these problems can be tracked to the issues up front.
The NAO has done a deep dive into why government IT projects fail, and she shared the key findings with the eForum:
“The first area we looked at was understanding the aims, the ambitions, and the risks. This is a big problem because there are generalists in government. I'm not talking about digital specialists here, who make a lot of the decisions about the scope and outcomes of major digital programmes – when they are not really qualified to do that.
I’m talking about unrealistic ambitions for untested technologies. When implemented at scale that’s asking for a huge amount of risk. It’s just not good practice. It's not even a question of risk management; it's more one of is this undertaking, right at the beginning, even feasible? Over-ambition plus cutting-edge technology is a major problem.
The second area we looked at is government’s attitude to engaging commercial partners. We found this was problematic on all sides.
A serious issue in large-scale programmes is that before things start for real there is insufficient thinking and analysis about architecture and design. Indeed, often this is skipped over completely. Suppliers are often asked to commit to contracts without a good understanding of the requirements, but detail and complexity always emerge over time.
The next is legacy systems and data. Phasing out legacy systems is a difficult and risky undertaking because of the complexities and interdependencies involved. A legacy system may be in the middle of a Spaghetti Junction of 50 interdependencies, so it’s not a trivial task.
This government has huge ambitions for its data. It wants to join it up and improve customer services, but that data – and therefore the data governance – resides in legacy systems. And the data itself is often old, of poor quality and hard to join up. The data and the legacy systems it’s in are completely bound together.
The next thing we looked at was capability. We have non-specialist business leaders in government, and we've got technical specialists. But the internal capability in government to deliver ambitious digital programmes needs to be seriously looked at.
In government, of course, there are a lot of highly experienced CIOs, but they sometimes just aren't listened to because the non-digital leader can't understand what the CIO is telling them about the risks or complexity involved. Plus, globally there is a hot market for digital skills, so getting them into government and retaining them is a whole other issue.
The trend for agile development sits uneasily atop these problems, she added:
Agile can't help if programmes are set up with the difficulties we've talked about already. Under the right circumstances, where it's appropriate to use it, agile is an important delivery method. But it needs to be done properly, with appropriate rigour and discipline. […] Overall, if the design isn't done properly up front, then no iteration will be able to change that at the end.
Of course, another key challenge in government is funding – especially in a tough economic environment that includes the pandemic, Brexit, and (alarmingly) the looming threat of war. As if all this wasn’t enough, there are structural challenges relating to government and money, she explained:
In government, the rules are very much focused on major infrastructural programmes for road building and bridge building. But as we know, the investment processes and risks associated with digital programmes are different. If you're going to build a road, you're going to build that road – it's been done over and over again. That doesn't mean there's no risk involved, but it’s not the same as the different phases of a digital programme.
To sum up, we recognise that addressing these challenges is difficult, but government really does need to start tackling them.
There’s no doubt that central government faces inherent challenges, notably constant disruptive change and upheaval, both within and between different administrations.
From time to time, digital champions have emerged at the top of UK government – such as Tony Blair, Peter Mandelson, Francis Maude, or Jeremy Hunt - and sometimes they emerge in the rat-run of special advisors along the corridors of power, such as Dominic Cummings, in his case dealing with a Cabinet that remains ill-matched with the digital world.
Change is so rapid at present that real progress – such as the UK’s Industrial Strategy, which underpinned investment in a range of new technologies – is often torn up just as it is getting started. We now have the ‘Plan for Growth’ and a raft of new strategies within it.
Local government, however, has greater continuity, especially in constituencies that have changed little over decades. Therefore, many expert CIOs have been able to make a name for themselves in local authorities. Transplanting comparable expertise into Whitehall and embedding it would seem to be essential.
But there’s no mechanism for doing so, not least because DCMS (as my previous article explained) is just not fit for purpose.