British government (once again) strives for creating a ‘data quality culture’

This week the British government has launched its Data Quality Framework, in a bid to create a ‘data quality culture' across the Civil Service. For those of us that have been following the digital and data work across the UK public sector for a while may well raise a cynical eyebrow at the announcement. Not because the substance of the framework isn't worthy or decent, but because it's very easy to get a sense of deja vu when thinking about data quality commitments in government.

You can look back as far as 2015 to see that senior officials were talking up the need for more effective data management in Whitehall, with talk of creating a ‘data-as-a-service' model and the need for more effective data sharing principles. However, what may be different this time around is that the COVID-19 pandemic has proven that access to quality data can really drive different outcomes.

Whitehall departments have been forced (and allowed, in some respects) to open up their walled gardens and share data in an attempt to quickly deliver COVID-19 relief services. Not only this, but you can easily see how access to data has made it possible (with varying degrees of success) to manage the pandemic in different parts of the country.

In other words, maybe this concept of ‘data quality culture' has finally sunk in for those right at the top of the Civil Service, who now realise that the government could be more effective if it pursued this agenda with more force.

Commenting on the launch of the Data Quality Framework, which was a key objective of the National Data Strategy, Professor Sir Ian Diamond, National Statistician, and Alex Chisholm, Chief Operating Officer for the Civil Service, said:

We find ourselves living in a society rich with data and the opportunities presented by this. In such an age, it is essential that public bodies have confidence that the data they access and process is fit for its intended purpose. Government's ambitions around digital transformation of public services and the UK becoming a world leader on AI are predicated on access to good quality data to inform decision-making and service delivery.

Yet concerns have been raised over the quality of data collected, created and used by government. Poor quality data in government leads to failings in services provided, poor decision-making, and an inability to understand how to improve. The2019 Public Accounts Committee Report (PDF, 303KB)showed that data has not been treated as an asset, and how it has become normal to ‘work around' poor-quality, disorganised data.

The extent of the data quality problem within government is poorly understood. Work on data quality is often reactive and not evidence-based. Where quality problems have been identified, the symptoms are often treated instead of the cause, leading to ineffective improvements and wasted resources.

Government needs a more structured approach to understanding, documenting and improving the quality of its data. This framework provides that, through data quality work that is proactive, evidence-based and targeted. It presents a set of principles for effective data quality management, and provides practical advice to support their implementation. While there is no such thing as ‘perfect quality' data, we must strive for a culture of continuous improvement. All public servants should understand why data quality is important, and feel able to proactively identify and address data quality issues.

The framework

The need for a data quality framework in government is clear. As the document released this week states, data is fundamental to effective, evidence-based decision-making. However, this is made harder - and riskier - if that data is of unknown or questionable quality. Poor or unknown quality data weakens evidence, undermines trust, and ultimately leads to poor outcomes.

The point of the framework is to ensure that data across government is fit for purpose. The level of quality, the document states, will depend on the purpose. Currently the process for managing data across government is inconsistent - which the framework aims to rectify.

It draws on "international and industry best practice" and sets out a series of principles, practices and tools aimed at achieving fit for purpose data. It's worth reading the framework in full, but we will outline the key elements below.

The first part of the framework provides a structure for organisations and individuals to frame their thinking around:

• Data quality principles - these include: committing to data quality; knowing your users and their needs; assessing quality throughout the data lifecycle; communicating data quality clearly and effectively; and anticipating changes affecting data quality.

• A guide to the data lifecycle to help organisations to identify and mitigate potential data quality issues at all stages

• Data quality dimensions, against which regular assessments of data quality can be made. These dimensions include: completeness, uniqueness, consistency, timeliness, validity and accuracy.

The second part of the framework provides guidance on practical tools and techniques, which can be applied to assess, communicate and improve data quality. These include:

• Data quality action plans - practical steps to assess quality data and make targeted improvements.

• Root cause analysis to ensure data quality work addresses issues at source

• Metadata guidance to support better use of metadata to communicate and interpret quality

• Communicating quality guidance, including suggested approaches for clearly communicating quality to users

• An introduction to data maturity models, for those who want to take a holistic approach to assessing and improving data quality

My take

This work is welcome and the initial impression is that the framework is a solid piece of documentation that could help guide and support organisations thinking about the quality of their data. However, what this will require is also intense leadership and strategic vision from those at the top of the Civil Service to make this a high priority. Working with data is incredibly difficult and the Civil Service is a fragmented, complex beast. If the desire is there to make better use of data for decision making, service delivery and policy design, then it needs to be put front in centre by those in charge.