An Old Challenge in the New Era: How the Public Sector Can Benefit from the Age of AI

Executive Summary

The Opportunity and the Challenge

Artificial Intelligence (AI) presents a significant opportunity to enhance the productivity and value of public services by removing information-intensive bottlenecks from delivery chains. However, the public sector’s history of costly and high-profile failures in technological implementation—from the NHS National Programme for IT to the Post Office Horizon scandal—demonstrates that realising this potential is not guaranteed.

The Core Argument: Capability, Not Technology

This report argues that the primary barrier to successfully leveraging AI is not the technology itself, but a persistent lack of foundational capabilities within public sector organisations and the wider innovation ecosystem. This challenge is compounded by a political economy that often incentivises short-term, highly visible ‘shiny projects’ over the patient, long-term investment required for genuine capability-building and sustained productivity growth.

A Diagnostic Framework for Action

To address this, the report introduces a diagnostic framework of twelve essential capabilities required to find and take advantage of the opportunities presented by AI. These are divided into two levels of action:

Part 1: Five Organisational Capabilities:

These are within the control of an individual organisation’s leadership and are crucial for successful project execution. They include:

• Organisational Learning: Systematically diagnosing root-cause problems in the service delivery chain.
• Planning & Prioritisation: Focusing resources on resolving true bottlenecks rather than superficial symptoms.
• Developing New Ways of Working: Integrating new technology with human capital and redesigned workflows.
• Managing Risk: Building the institutional resilience to learn from failure and foster intelligent experimentation.
• Managing Supplier Relations: Acting as an ‘intelligent customer’ to co-develop fit-for-purpose solutions and avoid vendor lock-in.

Part 2: Seven Sector-Level Capabilities

These are capabilities that strategic institutions (e.g., central government departments) must exercise to create a fertile ecosystem for innovation.

• Strengthening Innovation Networks: Connecting public bodies, researchers, and suppliers to share knowledge and prevent duplicated effort.
• Providing Clarity on Collaborative Performance: Reducing the risks of public-private partnerships through transparent frameworks and supplier vetting.
• Funding & Coordinating Research: Investing in high-risk, high-reward R&D (like explainable AI) that benefits the entire sector.
• Governing Failure: Designing governance that balances accountability with the need to learn from failure, encouraging honest reporting and intelligent risk-taking.
• Maintaining Pressure for Improvement: Using benchmarks and funding to create stable financial pressure that incentivises a search for efficiency.
• Ensuring a Flow of Human Capital: Addressing sector-wide skills gaps by funding training and creating robust professional standards.
• Standardising & Scaling Best Practice: Promoting crucial technical standards (e.g., for data interoperability) to create a common foundation for innovation.

Conclusion and Key Policy Implication

The value of AI for the public sector is not a given; it must be earned through deliberate, sustained effort. The central conclusion is that the successful adoption of AI is conditional on the strength of these twelve foundational capabilities. Without them, investment in AI will likely replicate past failures, leading to wasted resources and deepening public cynicism. The key policy implication is therefore to shift the focus from technological acquisition to organisational and systemic development. Leaders should use this framework to first diagnose their organisational and systemic weaknesses and then make targeted investments in these foundational capabilities. Leaders must prioritise building this foundation; without it, the profound promise of artificial intelligence may remain unrealised.

Author Joel Hoskins