A Global Comparison of Schemas, Transparency, and Interoperability in Public-Sector AI Registers and Inventories

Problem

This preprint addresses the gap in capability and representation of public-sector AI within existing registers and inventories. The authors identify a lack of comprehensive data on AI systems, which hampers transparency and interoperability across jurisdictions.

Method

The study analyzes a dataset comprising 8,368 records sourced from country-specific and transnational inventories across 72 countries. The authors examine 23 harmonized fields within these records to assess the completeness and consistency of the information provided. Key findings indicate that while the registers share a descriptive core, they are deficient in critical areas such as appeals processes, risk assessments, legal bases, and external evaluations.

Results

The analysis reveals substantial schema missingness across the registers, indicating that many fields are inadequately populated. Furthermore, the study finds no significant patterned convergence in schema similarity among the inventories, suggesting a lack of standardization. Additionally, the overlap of sources covering the same jurisdictions is selective, further complicating the landscape of public-sector AI documentation. The available text does not report quantitative results.

Limitations

The authors flag that the broad schemas analyzed contain significant missingness, which undermines the reliability of the data. The lack of significant patterned convergence in schema similarity is another limitation, indicating that efforts to harmonize data across registers have not been successful. These issues may affect the overall utility of the registers for stakeholders seeking to understand public-sector AI deployments.

Why it matters

The implications of this work are critical for downstream efforts aimed at improving transparency and interoperability in public-sector AI. By highlighting the deficiencies in current registers, the study underscores the need for standardized frameworks that can facilitate better data sharing and evaluation of AI systems. This could lead to enhanced governance and accountability in the deployment of AI technologies in the public sector.