Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video

Problem
This paper addresses the limitations of existing camera-controlled video generation methods, which typically require extensive training on large-scale camera-annotated datasets or involve complex post-training optimizations. The authors highlight the gap in the literature regarding training-free approaches that can effectively generate videos aligned with specified camera trajectories without necessitating additional denoising or optimization steps. This work is presented as a preprint and has not yet undergone peer review.

Method
The authors introduce “Warp-as-History,” a novel interface that transforms camera-induced warps into a pseudo-history of camera-warped frames, maintaining target-frame positional alignment and selecting visible tokens. The method operates without any modifications to the underlying video generation architecture or the need for test-time optimization. The core mechanism involves constructing a pseudo-history from past observations based on a specified camera trajectory, which is then processed through the model’s visual-history pathway. The positional encoding of the pseudo-history is aligned with the target frames being denoised, and tokens that do not correspond to valid source observations are discarded. Additionally, the authors propose a lightweight offline LoRA (Low-Rank Adaptation) fine-tuning approach that utilizes only one camera-annotated video to enhance the model’s performance, allowing it to generalize to unseen videos.

Results
The proposed method demonstrates significant improvements over baseline models on various benchmarks. The authors report enhanced camera adherence, visual quality, and motion dynamics, achieving a non-trivial zero-shot capability in following camera trajectories. While specific numerical results and effect sizes are not detailed in the abstract, the extensive experiments conducted on diverse datasets confirm the effectiveness of Warp-as-History compared to existing methods that rely on extensive training or optimization.

Limitations
The authors acknowledge that their approach, while effective, may still be limited by the quality and diversity of the single camera-annotated video used for fine-tuning. Additionally, the method’s performance in highly dynamic scenes or with complex camera movements is not explicitly addressed. The lack of peer review may also imply that the findings should be interpreted with caution until validated by the community.

Why it matters
The implications of this work are significant for the field of video generation, particularly in applications requiring real-time camera control and adaptability to varying input conditions. By demonstrating a training-free method that can effectively generate videos aligned with camera trajectories, this research opens avenues for further exploration in zero-shot learning and efficient video synthesis. The lightweight fine-tuning approach also suggests potential for practical applications in scenarios where annotated data is scarce, thereby broadening the accessibility of advanced video generation techniques.

Authors: Yifan Wang, Tong He
Source: arXiv:2605.15182
URL: https://arxiv.org/abs/2605.15182v1