Refuse, Decompose, Refresh: A Claim-Safe Protocol for Closed-Loop AI Evaluation

Problem

Closed-loop AI evaluation can inadvertently support incorrect claims, even when reproducibility is maintained. This paper addresses the need for a robust protocol that ensures claims made by AI systems are valid and reliable, particularly in the context of evolving data distributions. The work is presented as a preprint and has not undergone peer review.

Method

The authors propose a three-action protocol: Refuse, Decompose, and Refresh.

• Refuse: The system abstains from evaluation if the reference stream or runtime comparison lacks adequate support.
• Decompose: The evaluation process reports execution metrics, false admissions, and hypotheses separately to provide clearer insights into performance.
• Refresh: The protocol treats alarms related to distribution shifts as triggers to invalidate and recompute reference maps, ensuring that the evaluation remains relevant to current data conditions.

The evaluation is conducted using a simulator comprising 24 policy components, across 3 demand regimes and 2 fault-mask families. The development utilizes independent seeds, with a total of 1,440 heldout cases and 21,600 partition rows.

• Admission Results: The protocol achieved a reference admission rate of 55 out of 72 units and a runtime admission rate of 54 out of 55 units.
• False Admission Rate: The protocol recorded a false admission rate of 0 out of 20 represented components, with a one-sided exact 95% upper bound of 0.1391 under a frozen 0.20 rule.
• Negative-Log-Likelihood Difference: The difference was measured at 0.1264 nats per row across 540 unit-arm rows in 20 component clusters, with a 95% component-cluster interval of [0.0593, 0.1918].

Results

The results indicate a reference admission rate of 55/72 and a runtime admission rate of 54/55, with a stable false admission rate of 0/20. The negative-log-likelihood difference was reported as 0.1264 nats per row. Additionally, the protocol effectively triggered drift log alarms, with clean fault-null streams activating alarms in 15/15, 0/15, and 14/15 cases across the three demand regimes. The available text does not report quantitative results for any named baselines.

Limitations

The authors do not explicitly state limitations; however, potential concerns include the reliance on specific reference streams and the context in which distribution shifts occur, which may affect the generalizability of the protocol.

Why it matters

This work has significant implications for the development of AI systems, particularly in ensuring that claims made by these systems are substantiated and reliable. By implementing a structured protocol for closed-loop evaluation, researchers and engineers can enhance the integrity of AI evaluations, thereby fostering trust and accountability in AI applications.