Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair

Problem
The paper addresses the limitations of autoregressive chain-of-thought (CoT) reasoning in large language models (LLMs), which is inherently unidirectional. This unidirectional inductive bias leads to error snowballing, where an initial mistake propagates through subsequent reasoning steps, compromising the overall output. The authors highlight the need for a mechanism that allows LLMs to correct errors in reasoning by leveraging both past and future context. This work is a preprint and has not undergone peer review.

Method
The authors introduce Teleological Reasoning Infilling (TRI), a training framework designed to imbue decoder-only transformers with goal-conditioned bridging capabilities. The core innovation is the reframing of erroneous reasoning segments as fill-in-the-middle (FIM) tasks. Given a verified prefix premise (P), a verified downstream milestone (S), and the original query (Q), the model synthesizes a logical bridge (M) that connects (P) to (S). To facilitate this, the authors propose a Prefix-Suffix-Middle (PSM) sequence rearrangement, incorporating three non-overlapping sentinel tokens that allow (M) to attend to both (P) and (S) without altering the self-attention mechanism. The training process consists of two stages: (i) Supervised Fine-Tuning (SFT) on symbolically verified ((P, S, M)) triples sourced from formal mathematics corpora, and (ii) Direct Preference Optimization (DPO) using a deterministic symbolic verifier (Lean 4/Python) as the reward oracle, which mitigates LLM-judge sycophancy.

Results
TRI demonstrates state-of-the-art performance across three benchmarks, significantly outperforming existing methods. Notably, it achieves a reduction in per-problem token expenditure by 31.2%. The specific benchmarks and baseline models are not detailed in the abstract, but the results indicate a substantial improvement in reasoning accuracy and efficiency.

Limitations
The authors acknowledge that TRI’s reliance on a deterministic symbolic verifier may limit its applicability in more complex, less structured reasoning tasks. Additionally, the framework’s performance is contingent on the quality of the training data, which is drawn from formal mathematics corpora. The authors do not discuss potential scalability issues or the generalization of TRI to other domains outside formal reasoning.

Why it matters
The introduction of TRI has significant implications for enhancing the robustness of LLMs in reasoning tasks, particularly in applications requiring high accuracy, such as automated theorem proving and complex problem-solving. By enabling models to correct their reasoning dynamically, TRI could pave the way for more reliable AI systems in critical domains. This work contributes to the ongoing discourse on improving LLM capabilities and is relevant for future research in error correction and reasoning in AI, as published in arXiv.