COGENT: Continuous Graph Emulators with Neural Ordinary Differential Equations for Long-Term Physical Forecasting

Problem
This paper addresses the limitations of existing forecasting models in handling long-term predictions on irregular geospatial meshes, particularly in the context of physical simulations. Current methods often rely on fixed temporal discretization, which restricts their ability to generate predictions at arbitrary future times. Additionally, the authors highlight the need for improved stability in long-range forecasting, a gap that is particularly evident in autoregressive graph-based models. This work is presented as a preprint, indicating that it has not yet undergone peer review.

Method
The core contribution of COGENT is the integration of a graph-based history encoder with Neural Ordinary Differential Equations (ODEs) to model the dynamics of physical systems. The architecture consists of a history encoder that processes a finite history of system states and external forcings, producing node-wise context vectors that encapsulate both spatial interactions and temporal evolution. These context vectors serve as initial conditions for a latent Neural ODE, which is driven by interpolated future forcings and a relative rollout time parameter. The model employs a residual decoder to map the latent trajectories back to future physical states, facilitating direct multi-step forecasting without the need for iterative state feeding. To enhance training stability, the authors introduce rollout-horizon sampling and a progressive rollout-horizon scheduling strategy, which are critical for managing long-horizon supervision.

Results
COGENT was evaluated on transient ice-sheet simulations generated by the Ice-sheet and Sea-level System Model. The results demonstrate that COGENT achieves improved long-range stability compared to autoregressive graph baselines, although specific numerical results and effect sizes are not detailed in the abstract. The authors suggest that their approach outperforms existing methods in terms of stability and flexibility, particularly for long-term forecasting tasks.

Limitations
The authors acknowledge that while COGENT shows promise, there are inherent challenges in training Neural ODEs, particularly regarding the sensitivity to hyperparameters and the potential for overfitting with complex physical systems. Additionally, the reliance on a specific type of graph-based representation may limit the model’s applicability to other types of data structures. The paper does not address the computational cost associated with training and inference, which could be significant given the continuous nature of the model.

Why it matters
The implications of this work are significant for the field of physical forecasting, particularly in applications requiring stable long-horizon predictions on irregular geospatial meshes. COGENT’s ability to generate predictions at arbitrary future times could enhance the modeling of complex physical systems, such as climate dynamics and environmental changes. This research contributes to the growing body of work on continuous-time models in machine learning, as highlighted in related literature, and offers a promising direction for scalable physical forecasting methodologies, as published in arXiv cs.LG.