Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets

Problem
This preprint addresses the gap in the capability of large language models (LLMs) to effectively represent and analyze numeric tabular datasets, which are prevalent in scientific research. Existing methodologies either focus on predictive modeling for individual datasets, necessitating a uniform set of variable definitions, or fail to provide interpretable mechanisms for cross-dataset alignment. The authors aim to create a framework that allows for meaningful representation and comparison of heterogeneous numeric datasets without requiring shared variable names or conventions.

Method
The proposed methodology involves several key components:

1. Exploratory Data Analysis (EDA) Descriptors: Numeric tabular datasets are characterized using structured EDA descriptors that summarize statistical properties.
2. Embedding: These descriptors are embedded into a shared vector space utilizing a pretrained sentence transformer, facilitating the representation of diverse datasets in a unified format.
3. Canonical Correlation Analysis (CCA): Cross-dataset similarity is quantified through CCA, with a penalized version applied to recover sparse, interpretable variable-level correspondences. This approach identifies which statistical descriptors drive alignment between datasets.
4. Differential Privacy: An optional differential privacy mechanism is integrated into the descriptor set prior to embedding, allowing for secure deployment in sensitive contexts without needing access to raw data during comparisons.
   The methodology is evaluated across 15 datasets, including general-purpose benchmarks and specialized domains like materials informatics.

Results
The framework achieves a P@1 score of 0.9, indicating high accuracy in nearest-neighbor retrieval tasks. The robustness of cluster structures and retrieval performance is maintained across various embedding ablations and differential privacy budgets. These results suggest that the proposed method effectively captures the underlying relationships between heterogeneous numeric datasets, outperforming traditional approaches that lack interpretability and cross-dataset alignment capabilities.

Limitations
The authors acknowledge that the methodology may be limited by the quality and representativeness of the EDA descriptors used. Additionally, while differential privacy is an option, its implementation may introduce trade-offs in the fidelity of the embeddings. The paper does not address potential scalability issues when applied to very large datasets or the computational overhead associated with the embedding and CCA processes.

Why it matters
This work has significant implications for the integration of numeric tabular data into retrieval-augmented generation pipelines, enhancing the ability to perform data-driven algorithm selection and simulation model initialization for previously unknown datasets. By providing a principled approach to cross-dataset alignment and interpretability, the framework opens avenues for more effective utilization of diverse datasets in machine learning applications, particularly in fields where numeric data is predominant.

Authors: M. Ross Kunz, John Merickel, Keith Wilson
Source: arXiv:2605.30289
URL: https://arxiv.org/abs/2605.30289v1