A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models

Problem
This tutorial addresses the gap in understanding the theoretical foundations of diffusion models by framing them within the context of differential equations. It is particularly relevant as it is a preprint and unreviewed work, which may lack the rigor of peer-reviewed literature. The authors aim to clarify the connections between forward and reverse processes in diffusion models, which are often treated in isolation in existing literature.

Method
The authors develop diffusion models starting from the conditional Gaussian forward process, demonstrating that it can be represented using both ordinary differential equations (ODEs) and stochastic differential equations (SDEs). They derive marginalized forward ODE and SDE formulations that transition the data distribution ( p_0 = p_{\mathrm{data}} ) to a Gaussian prior ( p_1 = \mathcal{N}(0,I) ). The reverse dynamics are then formulated, leading to the reverse SDE and reverse probability-flow ODE, both governed by the marginal score ( \nabla \log p_t(x) ). The tutorial establishes a training objective for score estimation, revealing that the conventional noise-prediction objective is equivalent to score matching, differing only by an additive constant independent of model parameters. The authors also discuss various sampling methods for the learned reverse dynamics, including DPM-Solver, classifier guidance, and classifier-free guidance. They compare the Denoising Diffusion Probabilistic Models (DDPM) and Denoising Diffusion Implicit Models (DDIM) within this framework, showing that both share the same training objective, with DDPM sampling corresponding to discrete reverse-SDE sampling and DDIM sampling to reverse-ODE sampling.

Results
While the paper does not present empirical results or quantitative comparisons against established baselines, it provides a theoretical framework that unifies various diffusion model approaches. The authors emphasize the equivalence of training objectives across different methods, which could lead to improved understanding and potential optimizations in model training and sampling strategies.

Limitations
The authors acknowledge that their work is primarily theoretical and does not include empirical validation of the proposed methods. Additionally, the tutorial does not explore the computational efficiency of the derived methods in practical applications, nor does it address potential limitations in scalability or robustness of the models in real-world scenarios. The lack of experimental results may limit the immediate applicability of the theoretical insights.

Why it matters
This tutorial is significant as it provides a comprehensive theoretical foundation for diffusion models, which are gaining traction in generative modeling. By framing diffusion processes in terms of differential equations, it opens avenues for further research into more efficient training and sampling methods. The insights gained from this work could inform future developments in generative models, potentially leading to advancements in areas such as image synthesis, audio generation, and other applications where diffusion models are employed.

Authors: Jiayi Fu, Yuxia Wang
Source: arXiv:2605.22586
URL: https://arxiv.org/abs/2605.22586v1