Where Should a Document Live: Context, Representations, or Parameters?

Problem

Large language models (LLMs) often struggle to access new information necessary for answering questions that fall outside their pre-training data. This paper addresses the gap in capability regarding how LLMs can effectively incorporate external knowledge, particularly in knowledge-intensive tasks. The work is presented as a preprint and has not undergone peer review.

Method

The authors compare two adaptation methods for LLMs: representation-based methods, specifically using a key-value (KV) cache, and parametric methods that involve fine-tuning. The evaluation is conducted across five knowledge-intensive benchmarks, focusing on the accuracy of these methods under various storage budgets. The authors introduce an oracle setting where cartridges (KV) serve as the knowledge injection method. Performance metrics are centered on accuracy, particularly in scenarios with different compression rates.

Results

The results indicate that cartridges (KV) outperform parametric methods by 10 percentage points in accuracy. In terms of compaction performance, cartridges match the accuracy of parametric methods only at low compression rates, while they lag behind by 10 points when compression exceeds $50 imes$. In a multi-document retrieval scenario, cartridges lead parametric methods by 29 points and outperform compaction by 15 points when compared to in-context learning (ICL). However, the authors note a significant issue with catastrophic forgetting, reporting a 6% performance degradation on control benchmarks and a 13% drop in coding tasks when using cartridges.

Limitations

The primary limitation highlighted by the authors is the phenomenon of catastrophic forgetting associated with cartridges, which leads to performance degradation in certain tasks. This degradation is particularly pronounced in coding-related benchmarks. The authors do not discuss other potential limitations, such as the scalability of the proposed methods or the generalizability of the results across different LLM architectures.

Why it matters

This work has significant implications for the development of LLMs, particularly in enhancing their ability to integrate external knowledge effectively. By demonstrating the advantages of representation-based methods over traditional parametric approaches, the findings could inform future research on knowledge retrieval mechanisms in LLMs, potentially leading to more robust models capable of handling a wider range of queries.