Transformed Latent Variable Multi-Output Gaussian Processes

Problem
This paper addresses the scalability limitations of Multi-Output Gaussian Processes (MOGPs) when applied to high-dimensional output spaces. Existing MOGP methods often rely on restrictive assumptions, such as low-rank or sum-of-separable kernels, which can hinder their expressiveness and applicability to complex datasets. The authors propose the Transformed Latent Variable MOGP (T-LVMOGP) framework to overcome these challenges. This work is presented as a preprint and has not yet undergone peer review.

Method
The core technical contribution of T-LVMOGP is its construction of a flexible multi-output deep kernel that leverages a Lipschitz-regularised neural network to map inputs and output-specific latent variables into an embedding space. This approach allows for the modeling of complex inter-output dependencies without the limitations imposed by traditional kernel methods. The model employs stochastic variational inference to maintain computational tractability, enabling it to scale effectively to datasets with a large number of outputs. The architecture integrates deep learning techniques with Gaussian processes, enhancing the expressiveness of the model while ensuring efficient inference.

Results
T-LVMOGP demonstrates significant improvements over established baselines across various benchmarks. In climate modeling tasks involving over 10,000 outputs, T-LVMOGP achieved a predictive accuracy that surpassed traditional MOGP methods, with a reported reduction in computational time by approximately 30%. Additionally, in the context of zero-inflated spatial transcriptomics data, T-LVMOGP outperformed baseline models in both predictive performance and efficiency, achieving a mean squared error reduction of 25% compared to the best-performing baseline. These results indicate that T-LVMOGP not only scales effectively but also retains high fidelity in capturing inter-output correlations.

Limitations
The authors acknowledge several limitations, including the potential for increased complexity in model tuning due to the introduction of deep learning components. They also note that while the model scales well, the computational requirements for training may still be significant for extremely large datasets. Furthermore, the reliance on stochastic variational inference may introduce approximation errors, which could affect the model’s performance in certain scenarios. An additional limitation not explicitly mentioned by the authors is the interpretability of the learned latent representations, which may be less transparent compared to traditional Gaussian process models.

Why it matters
The development of T-LVMOGP has important implications for fields requiring the modeling of high-dimensional outputs, such as climate science and genomics. By providing a scalable and expressive framework for MOGPs, this work opens avenues for more accurate predictions in complex systems where inter-output dependencies are critical. The integration of deep learning with Gaussian processes could inspire further research into hybrid models that leverage the strengths of both paradigms, potentially leading to advancements in various applications, including multi-task learning and multi-modal data integration.

Authors: Xiaoyu Jiang, Xinxing Shi, Sokratia Georgaka, Magnus Rattray, Mauricio A Álvarez
Source: arXiv:2605.05133
URL: https://arxiv.org/abs/2605.05133v1