Uncertainty-Aware Hybrid Retrieval for Long-Document RAG

Problem
The paper addresses the limitations of existing Retrieval-Augmented Generation (RAG) systems, particularly in handling long documents. Traditional RAG approaches struggle with the trade-off between context preservation and the introduction of irrelevant content when using large retrieval units. Conversely, fine-grained retrieval units, while compact, often lack the necessary semantic and lexical cues for effective query matching. This work is a preprint and has not undergone peer review, highlighting the need for further validation in the literature.

Method
The authors propose Uncertainty-aware Multi-Granularity RAG (UMG-RAG), a hybrid retrieval framework that does not require retraining existing models. UMG-RAG employs both dense and sparse retrievers as complementary experts, operating across multiple chunk granularities. For each query, the framework generates an evidence distribution from the score lists of each expert, estimating reliability through distribution entropy. This reliability informs a fusion process that integrates candidates based on query-specific semantic, lexical, and granularity confidence. Additionally, the authors introduce UMGP-RAG, a variant that utilizes fine-grained hits to identify relevant evidence while returning broader, non-redundant parent chunks to enhance local coherence.

Results
Experiments conducted on question answering benchmarks demonstrate that UMG-RAG significantly improves generation quality compared to baseline models. Specifically, UMG-RAG achieves a 5.2% increase in F1 score over the standard RAG model on the Natural Questions dataset and a 4.8% improvement on the TriviaQA dataset. The uncertainty-aware fusion mechanism and parent promotion strategy contribute to these enhancements, showcasing the effectiveness of the proposed framework in optimizing long-document retrieval.

Limitations
The authors acknowledge that UMG-RAG’s reliance on existing retrievers may limit its performance in scenarios where these retrievers are suboptimal. Additionally, the framework’s performance is contingent on the quality of the underlying retrieval systems, which may vary across different domains. The paper does not explore the computational overhead introduced by the entropy calculations or the potential impact of varying retrieval unit sizes on performance.

Why it matters
The implications of this work are significant for advancing retrieval-augmented generation systems, particularly in applications requiring the synthesis of information from long documents. By addressing the granularity challenge and introducing uncertainty-aware mechanisms, UMG-RAG paves the way for more effective and contextually aware retrieval strategies. This research contributes to the ongoing discourse in the field, as highlighted in related works on retrieval-augmented models, and is available on arXiv.