When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

Problem

Large language models (LLMs) often generate fluent responses that lack robust factual support, leading to potential misinformation. This paper addresses the need for a mechanism that allows LLMs to selectively abstain from providing answers when uncertainty is high. The work is presented as a preprint and has not undergone peer review.

Method

The authors propose a novel framework called Chain-of-Self-Questioning (CoSQ), which consists of three evaluated variants: Grounded-CoSQ, Critical-CoSQ, and Adaptive-CoSQ. The framework is designed to enhance the decision-making process of LLMs by enabling them to self-question their responses before committing to an answer. The validation set used for evaluation is the 817-item TruthfulQA multiple-choice dataset. The authors tested eleven different open-weight and hosted model families to assess the effectiveness of the CoSQ framework. The final protocol employed is Grounded-CoSQ with a threshold parameter (τ) set to 0.90. The evaluation metrics include mean unconditional wrong-commitment rate, answered accuracy, and coverage.

Results

The results demonstrate significant improvements in the performance of LLMs using the CoSQ framework compared to traditional methods. The mean unconditional wrong-commitment rate achieved was 8.9%, a notable reduction from 13.1% observed with chain-of-thought prompting, resulting in a relative reduction of 32.1%. The answered accuracy was reported at 89.7%, surpassing the baseline of 86.9%. Coverage metrics for the different CoSQ variants were as follows: Grounded-CoSQ achieved 87.6%, Critical-CoSQ reached 88.6%, and Adaptive-CoSQ recorded 86.5%.

Limitations

The authors do not report any limitations in their study, and no obvious limitations are identified in the available text.

Why it matters

The introduction of the CoSQ framework has significant implications for the deployment of LLMs in real-world applications, particularly in scenarios where factual accuracy is critical. By enabling models to self-assess their confidence in responses, this approach could mitigate the risks associated with misinformation and enhance the reliability of LLM outputs. This work lays the groundwork for future research into adaptive response mechanisms in LLMs, potentially influencing the design of more robust AI systems.