Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use

Problem
This preprint addresses the gap in reliable execution of tool use by large language models (LLMs), which often struggle to balance reasoning depth with structural validity. Existing methods typically rely on raw outputs from exemplars, lacking a systematic approach to leverage historical execution trajectories. The authors propose a case-based framework, CAST, to enhance the reasoning capabilities of LLMs in tool use scenarios.

Method
CAST employs a case-driven approach that treats historical execution trajectories as structured cases. The framework extracts case-derived signals to identify complexity profiles, which inform optimal reasoning strategies, and failure profiles that highlight potential structural breakdowns. This knowledge is integrated into a fine-grained reward design for reinforcement learning, allowing the model to autonomously adapt case-based strategies. The architecture is not explicitly detailed, but the method emphasizes the use of historical data to inform adaptive reasoning and execution. The training process involves reinforcement learning, although specific compute resources are not disclosed.

Results
CAST was evaluated on two benchmarks: BFCLv2 and ToolBench. The framework achieved an overall execution accuracy improvement of up to 5.85 percentage points compared to baseline models. Additionally, it reduced the average reasoning length by 26%, which is significant in mitigating high-impact structural errors. These results indicate that CAST not only enhances schema-faithful execution but also improves task-level tool-use success, demonstrating a clear advantage over existing methods.

Limitations
The authors acknowledge that the reliance on historical execution cases may limit the generalizability of the approach to novel or unseen tasks. They do not address potential issues related to the scalability of the case extraction process or the computational overhead introduced by the fine-grained reward design. Furthermore, the framework’s performance in highly dynamic environments or with rapidly changing tool requirements remains untested.

Why it matters
The implications of this work are significant for the development of more reliable LLMs capable of tool use. By demonstrating that historical execution cases can provide reusable adaptation knowledge, CAST paves the way for future research into case-based reasoning in AI. This approach could lead to more robust LLMs that can autonomously adjust their reasoning strategies, ultimately enhancing their applicability in real-world scenarios where tool use is critical. The findings encourage further exploration of case-based frameworks in other domains of AI, potentially influencing the design of adaptive systems across various applications.

Authors: Renning Pang, Tian Lan, Leyuan Liu, Piao Tong, Sheng Cao, Xiaosong Zhang
Source: arXiv:2605.15041
URL: https://arxiv.org/abs/2605.15041v1