Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands

Problem
This position paper addresses a critical gap in AI governance frameworks that demand verifiable safety claims from AI systems, particularly between 2019 and early 2026. The authors argue that current behavioral assurance methodologies, which include behavioral evaluations and red-teaming, are insufficient for verifying safety claims related to latent representations and long-horizon agentic behaviors. This work is a preprint and has not undergone peer review, highlighting the urgency of the issues discussed.

Method
The authors introduce the concept of the “audit gap,” which represents the divergence between the safety claims required by governance frameworks and the verification capabilities of existing methodologies. They analyze a 21-instrument inventory to illustrate how geopolitical and industrial pressures create an incentive gradient favoring superficial behavioral proxies over deeper structural verification. The paper proposes a technical pivot to address these limitations by suggesting that the weight of behavioral evidence in legal texts should be bounded. Additionally, they advocate for extending voluntary pre-deployment access to include mechanistic-evidence classes, such as linear probes, activation patching, and before/after-training comparisons, to enhance the verification process.

Results
While the paper does not present quantitative results or benchmark comparisons typical of empirical studies, it provides a qualitative analysis of the limitations of current assurance methodologies. The authors emphasize that the existing frameworks fail to capture the complexities of AI systems, leading to potentially unsafe deployments. The identification of the audit gap serves as a conceptual framework for understanding the inadequacies in current verification practices.

Limitations
The authors acknowledge that their analysis is primarily theoretical and lacks empirical validation. They do not provide specific case studies or quantitative metrics to support their claims, which may limit the applicability of their conclusions. Additionally, the proposed solutions, while innovative, may face practical challenges in implementation, particularly in regulatory environments that are slow to adapt to new methodologies.

Why it matters
This work has significant implications for the future of AI governance and safety assurance. By formalizing the audit gap and introducing the concept of fragile assurance, the authors highlight the need for a paradigm shift in how safety claims are verified. The proposed technical pivot towards mechanistic evidence could lead to more robust verification processes, ultimately enhancing the safety and reliability of AI systems. This paper serves as a call to action for researchers and policymakers to rethink current assurance methodologies and develop frameworks that can adequately address the complexities of advanced AI systems.

Authors: Pratinav Seth, Vinay Kumar Sankarapu
Source: arXiv:2605.15164
URL: https://arxiv.org/abs/2605.15164v1