Learning Quantifiable Visual Explanations Without Ground-Truth

Problem
This paper addresses the challenge of evaluating explainable AI (XAI) methods due to the absence of reliable ground-truth explanations. Existing metrics for assessing explanation quality often fail to align with human intuitions, leading to a gap in the capability to quantitatively measure the effectiveness of XAI techniques. The authors propose a novel framework that provides a quantifiable metric based on continuous input perturbation, which is designed to better reflect the sufficiency and necessity of attributed information in relation to model decision-making. This work is presented as a preprint and has not yet undergone peer review.

Method
The core technical contribution is a new metric for evaluating XAI methods, which is derived from the principles of continuous input perturbation. This metric quantifies the quality of explanations by assessing how well the attributed information corresponds to the model’s decision-making process. The authors also introduce a novel XAI method that utilizes this metric as a differentiable supervision signal for fine-tuning a model. This approach involves the development of an adapter module that can be integrated with any black-box model, enabling it to generate causal explanations without compromising the model’s performance. The training process leverages the proposed metric to optimize the quality of the explanations produced.

Results
The authors demonstrate that their proposed method yields explanations that significantly outperform those generated by existing XAI techniques across various quantifiable metrics. While specific numerical results and benchmarks are not detailed in the abstract, the paper claims that the explanations align more closely with human intuitions compared to traditional metrics. The authors likely provide empirical evaluations against established baselines, although these details would need to be reviewed in the full paper for precise effect sizes and statistical significance.

Limitations
The authors acknowledge that their framework relies on the assumption that the perturbation-based metric accurately captures human intuitions about explanation quality, which may not universally hold across all contexts. Additionally, the method’s performance may vary depending on the complexity of the underlying model and the nature of the data. An obvious limitation not explicitly mentioned is the potential computational overhead introduced by the fine-tuning process, which may affect scalability in real-world applications.

Why it matters
This work has significant implications for the field of XAI, as it provides a systematic approach to quantifying explanation quality, which is crucial for the validation and responsible deployment of AI systems. By offering a method that can be integrated with existing models to produce causal explanations, the research paves the way for more interpretable AI systems that align better with human reasoning. This could enhance trust in AI applications across various domains, including healthcare, finance, and autonomous systems, where understanding model decisions is critical.

Authors: Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny, Dimosthenis Karatzas
Source: arXiv:2605.18681
URL: https://arxiv.org/abs/2605.18681v1