WorldOlympiad: Can Your World Model Survive a Triathlon?

Problem
The paper addresses a significant gap in the evaluation of video-based world models, particularly in assessing their physical faithfulness, geometric consistency, and interaction fidelity. Existing benchmarks primarily focus on visual quality, semantic alignment, or short-term temporal coherence, which do not adequately capture whether generated videos adhere to physical laws, maintain coherent 3D structures, or support controllable interactions over extended periods. This work is presented as a preprint, indicating that it has not yet undergone peer review.

Method
WorldOlympiad introduces a comprehensive evaluation framework that decomposes world-model assessment into three distinct tracks: physical, geometry, and interaction.

1. Physical Track: Utilizes object segmentation and a multi-layered language model (MLLM) as a judge to evaluate whether generated videos conform to interpretable physical rules, including mechanics, thermal phenomena, and material properties.

2. Geometry Track: Employs Gaussian splatting to reconstruct generated videos, assessing structural consistency, cross-view coherence, and alignment with camera trajectories.

3. Interaction Track: Tests the ability of generated rollouts to follow complex action prompts while ensuring smooth and coherent transitions across consecutive video segments.

The benchmark encompasses three major downstream scenarios: gaming, robotics, and general real-world videos, addressing diverse challenges such as interactive control, embodied manipulation, and open-domain motion dynamics.

Results
Experiments conducted on state-of-the-art generative models reveal significant deficiencies in physical reasoning, 3D consistency, and long-horizon interaction capabilities. The authors report that existing models struggle to meet the benchmarks set by WorldOlympiad, highlighting the inadequacy of current evaluation protocols. Specific quantitative results are not detailed in the abstract, but the emphasis on “substantial gaps” suggests a marked performance drop compared to baseline models.

Limitations
The authors acknowledge that the benchmark may not cover all possible scenarios or edge cases in world modeling, potentially limiting its applicability in certain contexts. Additionally, the reliance on MLLM for physical rule assessment may introduce biases based on the model’s training data and inherent limitations. The paper does not address the computational overhead associated with the proposed evaluation framework, which could impact its scalability in practical applications.

Why it matters
WorldOlympiad represents a critical advancement in the structured evaluation of generative world models, emphasizing the need for comprehensive assessments that go beyond visual fidelity. By exposing failure modes in physical reasoning and interaction fidelity, this benchmark can guide future research towards developing more robust and reliable world models. The implications of this work extend to various fields, including robotics and gaming, where accurate modeling of physical interactions is essential. This research is available on arXiv and sets the stage for future improvements in generative model evaluation methodologies.