Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template Constrained Decoding

Problem
This preprint addresses the gap in the reliability of Text-to-SQL systems, particularly in real-world applications where complex or unseen database schemas are prevalent. Despite advancements in large language models (LLMs) for natural language to SQL translation, existing methods often yield inconsistent accuracy and can generate invalid SQL queries. The authors identify the need for a systematic approach to improve the accuracy and efficiency of SQL generation by leveraging recurring query patterns in labeled datasets.

Method
The core technical contribution is the Template Constrained Decoding (TeCoD) framework, which consists of two main components: a template generation module and a grammar-constrained decoding mechanism. TeCoD first converts historical natural language (NL) and SQL pairs into reusable templates. A fine-tuned natural language inference model is employed to select the most appropriate template for a given query, ensuring that only relevant templates are considered. The SQL generation process is then guided by these templates through a partitioned strategy that enforces syntactic validity while optimizing for computational efficiency. The authors do not disclose specific training compute details but emphasize the use of historical data for template creation.

Results
TeCoD demonstrates significant improvements over in-context learning (ICL) baselines, achieving up to 36% higher execution accuracy on matched queries. Additionally, it reduces latency by 2.2 times compared to ICL methods. These results were validated on a benchmark dataset, although specific datasets and baseline models used for comparison are not detailed in the abstract.

Limitations
The authors acknowledge that the effectiveness of TeCoD is contingent on the availability of high-quality labeled NL-SQL pairs for template generation. They also note that the approach may struggle with highly dynamic schemas that deviate significantly from historical patterns. An additional limitation not explicitly mentioned is the potential for overfitting to the templates, which could reduce generalization to novel queries not represented in the training data.

Why it matters
The implications of this work are significant for the development of more reliable Text-to-SQL systems, particularly in enterprise applications where data schemas can be complex and variable. By introducing a structured approach to SQL generation that leverages historical query patterns, TeCoD could enhance the robustness of LLMs in practical deployments. This framework may also inspire further research into template-based methods for other natural language processing tasks, potentially leading to improved performance in areas where structured outputs are required.

Authors: Smit Jivani, Sarvam Maheshwari, Sunita Sarawagi
Source: arXiv:2604.28028
URL: https://arxiv.org/abs/2604.28028v1