Beyond representational alignment with brain-guided language models for robust reasoning

Problem
The paper addresses the insufficient characterization of the alignment between large language models (LLMs) and the neural mechanisms of human higher-order cognition, particularly in the context of deductive reasoning. The authors highlight a gap in the literature regarding whether LLMs can be improved by aligning their internal representations with neural signals from reasoning-related brain regions. This work is presented as a preprint, indicating that it has not yet undergone peer review.

Method
The authors propose a brain-guided framework that leverages neural-predictivity metrics to enhance LLM reasoning capabilities. They analyze the alignment of LLM internal representations with task-fMRI activity, revealing that while there is substantial aggregate alignment, specific reasoning types show lower predictivity. The framework involves steering model representations along directions induced by the joint structure of model and brain representations. This is achieved through interventions at inference and fine-tuning during training. The experiments were conducted across 10 LLMs, ranging from 1.5 billion to 72 billion parameters, utilizing task-evoked brain signals to enhance reasoning performance.

Results
The proposed framework yielded significant improvements in reasoning accuracy, achieving up to a 13% absolute gain across various reasoning types. The enhancements were shown to be orthogonal to traditional language-only supervision methods. The authors report that LLMs explain a substantial fraction of the explainable variance in reasoning-related brain regions at the aggregate level, indicating both alignment and divergence in specific reasoning tasks. These results were benchmarked against standard reasoning tasks, demonstrating the effectiveness of the brain-guided approach.

Limitations
The authors acknowledge that while their approach shows promise, the predictivity within specific reasoning types remains lower than desired, indicating areas for further exploration. Additionally, the reliance on task-evoked brain signals may limit the generalizability of the findings across different cognitive tasks or populations. The paper does not address potential ethical implications of using neural data to guide AI development, nor does it explore the scalability of the proposed framework to larger or more diverse datasets.

Why it matters
This research advances the understanding of LLM-brain correspondences from mere correlation to actionable guidance, suggesting a novel pathway for enhancing AI reasoning capabilities through cognitive alignment. The implications of this work extend to the development of more robust AI systems that can better mimic human reasoning processes, potentially influencing future research in cognitive computing and AI interpretability. This is particularly relevant for applications requiring high-stakes decision-making, where alignment with human cognitive processes is crucial, as published in arXiv cs.LG.