Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment

Problem
This position paper addresses a critical gap in the literature regarding the safety of Large Language Model (LLM) agents during deployment. The authors argue that existing frameworks, which attempt to enforce safety within a single abstraction layer, are fundamentally inadequate. This inadequacy is not merely a limitation of current systems but a structural issue inherent to the execution of LLM agents. The paper posits that safe operation requires a multi-layered approach that accounts for three distinct dimensions: semantic intent and policy compliance, environmental validity, and dynamical feasibility. The authors call for a contract-based architecture to ensure safety across these dimensions, highlighting that this is an unreviewed preprint.

Method
The proposed architecture consists of three layers, each independently certifying a specific safety dimension. The authors suggest that each layer operates under a probabilistic guarantee that satisfies the assumptions of the subsequent layer. This layered approach allows for a more robust safety framework, as it acknowledges the distinct information requirements and operational contexts of each safety dimension. The paper outlines a compositional system-level safety bound derivation using the chain rule of probability, which provides a theoretical foundation for the proposed architecture. However, the authors identify three significant open problems that need to be addressed for practical implementation: (1) bound estimation from non-i.i.d. traces, (2) graceful degradation of contracts under deployment drift, and (3) extension to multi-agent settings.

Results
As this is a position paper, it does not present empirical results or quantitative comparisons against existing baselines or benchmarks. Instead, it focuses on theoretical constructs and the necessity of a multi-layered safety architecture. The authors emphasize the importance of addressing the identified open problems to facilitate the deployment of LLM agents with guaranteed safety.

Limitations
The authors acknowledge that their proposed architecture is still conceptual and requires further development to be deployable. They specifically highlight the challenges of bound estimation from non-i.i.d. traces and the need for contracts to gracefully degrade under deployment drift. Additionally, the extension to multi-agent settings remains an unresolved issue. An obvious limitation not explicitly mentioned is the potential computational overhead introduced by the multi-layered architecture, which could impact real-time performance in practical applications.

Why it matters
This work has significant implications for the future of LLM agent deployment, particularly in safety-critical applications. By advocating for a contract-based architecture that enforces safety across multiple dimensions, the authors provide a framework that could enhance the reliability and trustworthiness of LLM agents. Addressing the outlined open problems is crucial for advancing the field and ensuring that LLM agents can operate safely in dynamic environments. This position paper serves as a call to action for researchers and practitioners to develop methodologies that can effectively implement the proposed safety architecture.

Authors: S. Bensalem, Y. Dong, M. Franzle, X. Huang, J. Kroger, D. Nickovic, A. Nouri, R. Roy et al.
Source: arXiv cs.AI
URL: https://arxiv.org/abs/2605.18672v1
arXiv ID: 2605.18672