Call for Evidence: AI testing, evaluation and assurance in the UK

The NPL Centre for AI Measurement is seeking evidence to inform how it supports the UK's emerging AI assurance market and the development of rigorous, comparable methods for testing and evaluating AI systems. We are interested both in evidence about wider challenges across the UK's AI assurance ecosystem and, where relevant, your own organisation's needs and potential interest in future support or collaboration.

For the purpose of this call, AI assurance refers to the process of measuring, evaluating and communicating the trustworthiness of AI systems and their components.

This questionnaire focuses primarily on the technical foundations of assurance: the measurement science, methods, metrics, tools and infrastructure used to test and evaluate AI. Broader governance, legal, organisational and regulatory aspects are important, but are not the primary focus of this call. For more information, please visit the Call for Evidence page.

Part 1: About you

Are you responding on behalf of an organisation or in a personal capacity?

In a personal capacityOn behalf of an organisation

Which perspective best represents your primary interest in this call for evidence?

Primarily a supplier of assurance – developing and/or providing AI assurance products or servicesPrimarily on the demand side – needing, buying, or relying on AI testing and evaluation for assurancePrimarily an intermediary – for example a researcher, regulator, standards body, investor, funder, trade body or other ecosystem organisationYou answer will be used to tailor questions you see in Part 2.

Part 2: Challenges, unmet needs and opportunities

This section gathers evidence about the underlying challenges and unmet needs in AI testing and evaluation. Part 2A is for all respondents and asks about challenges across the wider UK ecosystem. Part 2B is optional and asks about needs, opportunities or interests specific to you or your organisation.

Part 2A: Ecosystem-wide challenges and capability gaps

Drawing on your experience to date, where do you see the most significant unmet needs or capability gaps in AI testing and evaluation for assurance?

Please identify, where you can, the types of AI, sectors, use cases, risk domains or evaluation capabilities affected.

What is preventing the market or existing ecosystem from addressing these gaps effectively?

Where could public intervention add the greatest value, without duplicating or displacing activity the market is already delivering?

Part 2B: Organisation-specific needs and opportunities

This section is optional and asks about your own organisation's needs or interests. Questions are tailored to the primary perspective you selected in Part 1.

Supply side

For organisations developing or providing AI testing, evaluation or assurance products and services.

What specific AI testing and evaluation capability would you most like to develop, validate or scale over the next 1–2 years, where public support could help?

Please describe the capability, the AI systems or use cases it would apply to, and the unmet market or user need it would address.

What currently prevents you from developing, validating or scaling this capability?

Please identify the most significant barriers, such as access to data, models, expertise, real-world testing environments, standards, validation or funding.

What form of public support or collaboration would make the greatest difference, and what practical outcome would it enable?

Demand side

For organisations developing, procuring or deploying AI

Describe an AI use case that you would like to pilot, procure, deploy or scale, but cannot currently proceed with confidently because of unmet AI testing, evaluation or assurance needs.

Please describe the use case, the type of AI system involved and its intended context of use, and how the testing or evaluation gap is affecting your ability to proceed.

What specifically are you unable to test, evaluate or verify with confidence?

What type of support would most help you address this gap and progress the use case?

Intermediary

For research, standards, regulatory or other ecosystem organisations

What expertise, evidence, or capability could you or your organisation potentially contribute to addressing the testing and evaluation challenges identified above?

For example, this could include research expertise, datasets, standards expertise or programme delivery.

Part 3: Intervention design

We are exploring a range of ways a publicly funded programme could support the development and piloting of scientifically rigorous, repeatable and comparable AI testing and evaluation techniques underpinning AI assurance. The types of interventions below are illustrative examples to prompt your thinking, not a fixed or final set. We would like to understand which you think would be most useful, which you might take part in, and what conditions would need to be in place. We would also like to hear about approaches we haven't listed.

a) Collaborative research sprints: a series of short, intensive events bringing assurance providers, deployers and researchers together to tackle a defined assurance challenge.

b) Matched testing pilot: pairing an assurance provider with a deploying organisation to conduct testing and evaluation of a deployment-ready AI use case, contributing towards codification of best practice.

c) Innovation challenge fund: a competitive grant where teams apply to solve a defined problem developed by a sponsor (e.g. governmental body or business) in testing and evaluation, with winning proposals funded to develop and trial their solution, contributing towards the codification of best practice

d) AI assurance methods accelerator: giving third-party AI assurance providers dedicated access to a network of technical experts from NPL and strategic partner organisations to workshop methodological challenges, refine and validate emerging assurance approaches and build evidence on their effectiveness, repeatability and limitations.

Which approach do you think would be most valuable overall and why?

Is there a different approach, not listed above, that you think would more effectively strengthen AI testing and evaluation?

Please describe it, including what it would produce.

Which, if any, would you or your organisation be interested in participating in and in what capacity?

For example, as an assurance provider, as a deploying organisation hosting a test, as a research or delivery partner, or as an advisor.

What specific conditions or support would your organisation need in order to participate?

Are there particular areas you think we should focus on as part of this programme?

For example, types of AI, AI testing and evaluation techniques, or risk domains.

Is there anything else you would like to share with us on this topic?

Part 4: Closing questions

NPL does not intend to publish individual responses to this Call for Evidence. We expect to publish an aggregated summary of the insights gathered, which may take the form of a report or a blog post. We will not attribute findings to individual respondents or organisations.

May we include your organisation, or your name if you're responding in a personal capacity, in a published list of contributors to this call for evidence?

YesNoThis would acknowledge that you contributed to the evidence base only. We will not attribute any particular views or findings to you as a result of your inclusion in this list.

May we contact you to clarify or discuss any part of your response?

YesNo

Would you like to be kept informed about the outcomes of this Call for Evidence and relevant next steps or future opportunities to engage with the Centre for AI Measurement?

YesNo

If you answered 'Yes' to either of the previous two questions, please provide your contact details:

Name:

Email address:

Your contact details will be used only for the purposes you have selected above and will be handled in accordance with the NPL Privacy Notice.