Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning

Problem
This paper addresses the gap in understanding the approximation and statistical complexity of learning multiple operators in a multi-task setting, specifically through the lens of the Multiple Neural Operators (MNO) architecture. The authors highlight that existing literature lacks comprehensive bounds on the performance of multi-task operator learning, particularly in terms of approximation and generalization capabilities. This work is presented as a preprint and has not yet undergone peer review.

Method
The core technical contribution of this paper is the derivation of near-optimal upper bounds for both approximation and statistical generalization in the context of Lipschitz multiple operator maps. The authors introduce the MNO architecture, which facilitates the learning of multiple operators simultaneously while sharing representations across tasks. They establish a curse of parametric complexity, leading to minimax rates that characterize the performance of the MNO. The authors also conduct a comparative analysis with a multi-task extension of DeepONet, which utilizes concatenated task inputs. They demonstrate that both architectures achieve similar asymptotic rates from a worst-case approximation-complexity perspective. The training compute requirements are not explicitly disclosed, but the theoretical framework suggests efficient scaling with respect to the number of tasks.

Results
The authors present theoretical results indicating that the MNO architecture achieves near-optimal rates for approximation and statistical generalization, matching the scaling laws observed in single operator learning. While specific numerical results are not provided in the abstract, the paper claims that the performance of MNO is comparable to that of the multi-task DeepONet architecture. The implications of these results suggest that the cost of learning multiple operators does not exceed that of learning a single operator, which is a significant finding for multi-task learning paradigms.

Limitations
The authors acknowledge that their results are primarily theoretical and may not directly translate to practical implementations without further empirical validation. They do not address potential limitations related to the scalability of the MNO architecture in real-world applications or the computational overhead associated with training multiple operators simultaneously. Additionally, the paper does not explore the impact of varying task complexities or the potential for overfitting in high-dimensional settings.

Why it matters
This work has significant implications for the field of multi-task learning, particularly in applications where learning multiple related tasks simultaneously can lead to improved efficiency and performance. By establishing that shared representations do not incur additional costs, the findings encourage further exploration of multi-task architectures in various domains, such as robotics, natural language processing, and computer vision. The theoretical insights provided could guide future research in optimizing multi-task learning frameworks and developing more robust operator learning methods.

Authors: Adrien Weihs, Hayden Schaeffer
Source: arXiv:2605.22724
URL: https://arxiv.org/abs/2605.22724v1