MatrixFormer: A Foundation Model for Matrix Completion

Problem

Existing tabular foundation models approach matrix completion as an entry-by-entry prediction task, neglecting the inherent two-dimensional structure of matrices. This paper presents MatrixFormer, a foundation model specifically designed to address this gap by leveraging the matrix's structural properties for improved completion performance. The work is a preprint and has not undergone peer review.

Method

MatrixFormer employs a matrix-native transformer architecture that is tailored for matrix completion tasks. The model is trained on synthetic low-rank and latent-factor matrices, incorporating diverse missingness patterns to enhance its robustness. The training mechanism involves a pre-trained model that predicts the full distribution for every missing entry in a single forward pass, allowing for efficient and comprehensive matrix completion.

Results

The available text does not report quantitative results. However, the authors claim that MatrixFormer demonstrates competitive performance across several tasks, including causal inference, language-model benchmark-score completion, tabular imputation, and recommendation systems matrix completion, when compared to existing models.

Limitations

The authors do not report any limitations in their work. However, the lack of quantitative results may hinder the ability to fully assess the model's performance relative to specific baselines.

Why it matters

The introduction of MatrixFormer has significant implications for downstream applications in matrix completion, particularly in scenarios where the two-dimensional structure of data is critical. By addressing the limitations of existing models, this work paves the way for more effective and efficient matrix completion techniques, potentially enhancing performance in various domains such as recommendation systems and data imputation.