National Data Library, the way forward

“We need to unlock access and break down silos”

Craig Suckling, Chief Data Officer

Unlocking the Potential of the NDL: Supporting the UK Government’s National Data Library Initiative

The UK Government’s National Data Library (NDL) initiative represents a bold vision for centralising, securing, and ethically leveraging public sector data to drive innovation. As part of the AI Opportunities Action Plan, the NDL aims to unlock the value of public sector data assets to support AI research and development while maintaining public trust in data sharing.

Clifford McDowell, founder of Doorda, wholeheartedly supports this initiative:

“The National Data Library can revolutionise how data is used in the UK. Doorda is proud to act as an AI enabler, delivering centralised, high-quality data to drive innovation and support the government’s ambitions.”

Overcoming Challenges to Build a Successful NDL

The government has previously attempted to establish centralised open data portals, but these efforts were hindered by a lack of funding and legal mandates. As a result, public sector organisations continued to publish fragmented datasets across individual websites, each with varying formats and naming conventions.

Before it can become a successful AI enabler, the NDL has several gaps it needs to bridge:

Comprehensive Data Aggregation
- Collect data from hundreds of public sector websites on a monthly rotation, ensuring a single source of truth for public sector datasets.
Preservation of Historical Data
- Many datasets that were removed or lost from public websites need to be restored to provide continuity and depth for the National Data Library (NDL).
Unified and Standardised Datasets
- Public sector data often exists in diverse formats due to varying IT systems. These datasets need to be standardised, creating consistent, unified datasets that are ready for use.
Terabytes of Data Ready for the NDL
- With terabytes of data available across 100s of public websites—and more in development— the NDL will need to curate and maintain these sources to create scalable, accurate, and high-potential datasets.

Supporting the AI Opportunities Action Plan

The AI Opportunities Action Plan emphasises the need for high-quality data to enable cutting-edge AI development. As Doorda’s datasets are derived from observational rather than modelled data, they are particularly valuable for training machine learning algorithms. These datasets reflect real-world patterns, making them ideal for applications in areas such as location analysis, urban planning, and business insights.

By monitoring government policies and local authority changes, Doorda ensures its datasets remain accurate and relevant, further positioning itself as an AI enabler for the NDL and the government’s broader AI goals.

Fostering Public Trust in Ethical Data Sharing

Craig Suckling, Chief Data Officer for the UK Government’s Central Digital and Data Office (CDDO), highlighted the importance of public trust:

“We need to unlock access and break down silos while maintaining public trust in secure and ethical data sharing across the public sector and nationally.”

Doorda supports these principles by adhering to stringent security and ethical standards, ensuring that its role as an AI enabler aligns with public sector expectations for transparency and accountability.

Enhancing the National Data Library

Doorda’s ability to support the development and curation of a National Data Library (NDL) go beyond aggregation and standardisation:

High-Potential Datasets for Innovation: Doorda actively identifies and delivers datasets that empower AI research and innovation.
Taxonomies and Data Engineering: Advanced taxonomies and data engineering techniques make public sector data commercially useful and easy to integrate into decision-making frameworks.

Conclusion

The UK Government’s National Data Library (NDL) initiative is a vital step toward transforming public sector data into a resource that drives AI innovation and national growth. With its expertise in data aggregation, preservation, and standardisation, Doorda is keen to support the government in achieving its AI Opportunities goals.

As an AI enabler, Doorda’s commitment to secure, ethical, and high-quality will hopefully aid in the success of the NDL, fostering innovation and creating a centralised resource for public and private sector stakeholders alike.