Chunking German Legal Code

Problem
This preprint addresses the gap in effective chunking strategies for retrieval-augmented generation (RAG) specifically within the context of German statutory law. The authors focus on the German Civil Code as a structured benchmark corpus, highlighting the need for optimized segmentation methods that enhance legal question-answering systems. Existing literature lacks comprehensive evaluations of various chunking techniques tailored to the unique structure of legal texts, which often contain complex hierarchical relationships.

Method
The authors implement and compare multiple chunking strategies, including:

• Structural Units: Segmentation based on sections, subsections, sentences, and propositions.
• Fixed-Size Windows: Uniformly sized segments regardless of content.
• Contextual Chunking: Segmentation informed by the context of the text.
• Semantic Clustering: Grouping based on semantic similarity.
• Lumber-style Chunking: A method that emphasizes semantic coherence.
• RAPTOR-based Hierarchical Retrieval: A retrieval approach that leverages hierarchical structures.

The evaluation is conducted on a legal question-answering dataset with section-level gold labels, measuring key performance indicators such as recall, query latency, index build time, and storage requirements. The authors emphasize the importance of aligning chunking strategies with the inherent legal structure to maximize retrieval effectiveness.

Results
The results indicate that chunking strategies that adhere to the legal structure, particularly those based on sections and subsections, achieve the highest recall rates. In contrast, more complex methods that deviate from this structure, such as contextual chunking and RAPTOR, demonstrate inferior performance. The authors report that simpler methods not only yield better recall but also exhibit favorable computational efficiency, with lower query latency and reduced index build times compared to LLM-intensive techniques. The findings suggest a significant trade-off between semantic enrichment and operational cost, underscoring the importance of preserving domain-specific structures in legal information retrieval.

Limitations
The authors acknowledge several limitations, including the potential overfitting of simpler chunking methods to the specific dataset used, which may not generalize across other legal texts or jurisdictions. They also note that while their methods improve recall, they do not address the precision of the answers provided. Additionally, the study does not explore the impact of varying the size of fixed-size windows or the potential benefits of hybrid approaches that combine multiple chunking strategies.

Why it matters
This work has significant implications for the development of legal information retrieval systems, particularly in jurisdictions with complex legal frameworks. By demonstrating that simpler, structure-aligned chunking methods can outperform more sophisticated techniques, the findings encourage future research to prioritize domain-specific characteristics in the design of retrieval systems. This could lead to more efficient and effective legal question-answering applications, ultimately enhancing access to legal information and supporting legal practitioners in their work.

Authors: Max Prior, Natalia Milanova, Andreas Schultz
Source: arXiv:2605.19806
URL: https://arxiv.org/abs/2605.19806v1