Atomic Fact-Checking Increases Clinician Trust in Large Language Model Recommendations for Oncology Decision Support: A Randomized Controlled Trial

Problem
This paper addresses the gap in understanding how different explainability techniques for AI recommendations impact clinician trust, particularly in the context of oncology decision support. The authors investigate whether atomic fact-checking—decomposing AI-generated treatment recommendations into individually verifiable claims linked to source guidelines—can enhance clinician trust compared to traditional explainability methods. This work is presented as a preprint and has not yet undergone peer review.

Method
The study employs a randomized controlled trial design involving 356 clinicians who provided 7,476 trust ratings for AI treatment recommendations. The core technical contribution is the implementation of atomic fact-checking, which contrasts with traditional transparency mechanisms. The atomic fact-checking approach links each recommendation to specific, verifiable claims derived from established clinical guidelines, allowing clinicians to assess the validity of each claim independently. The authors measure trust using Cohen’s d effect size to quantify the impact of the different explainability methods on clinician trust levels.

Results
The results indicate that atomic fact-checking significantly increases clinician trust, with a Cohen’s d of 0.94, suggesting a large effect size. The proportion of clinicians expressing trust in AI recommendations rose from 26.9% to 66.5% when using atomic fact-checking. In contrast, traditional transparency mechanisms yielded a smaller effect size, with Cohen’s d ranging from 0.25 to 0.50, indicating a dose-response gradient of improvement over baseline trust levels. These findings suggest that the atomic fact-checking method is substantially more effective in fostering trust among clinicians than conventional approaches.

Limitations
The authors acknowledge several limitations, including the potential for selection bias in the clinician sample and the specific context of oncology, which may not generalize to other medical domains. Additionally, the study does not explore the long-term effects of increased trust on clinical decision-making or patient outcomes. The reliance on self-reported trust ratings may also introduce subjective bias. Furthermore, the study does not address the computational overhead associated with implementing atomic fact-checking in real-world clinical settings.

Why it matters
This research has significant implications for the development of AI systems in healthcare, particularly in high-stakes environments like oncology. By demonstrating that atomic fact-checking can substantially enhance clinician trust, the findings suggest that integrating this approach into AI recommendation systems could lead to better acceptance and utilization of AI tools in clinical practice. This could ultimately improve decision-making processes and patient care outcomes. The study highlights the importance of explainability in AI, emphasizing that the manner in which information is presented can critically influence trust and, consequently, the effectiveness of AI in supporting clinical decisions.

Authors: Lisa C. Adams, Linus Marx, Erik Thiele Orberg, Keno Bressem, Sebastian Ziegelmayer, Denise Bernhardt, Markus Graf, Marcus R. Makowski et al.
Source: arXiv:2605.03916
URL: https://arxiv.org/abs/2605.03916v1