Diffusion Models

A denoising diffusion probabilistic model defines a fixed process that gradually corrupts data and learns a reverse process that reconstructs samples step by step.

Forward Process

For a variance schedule $\beta_t$ and $\alpha_t=1-\beta_t$,

$$q(x_t\mid x_{t-1}) = \mathcal{N}\!\left( x_t;\sqrt{\alpha_t}\,x_{t-1}, (1-\alpha_t)I \right).$$

Let $\bar{\alpha}_t=\prod_{s=1}^{t}\alpha_s$. Any timestep can be sampled directly from clean data:

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon, \qquad \epsilon\sim\mathcal{N}(0,I).$$

This closed form makes it unnecessary to simulate every earlier noising step during training.

Learned Reverse Process

Generation needs the reverse transition

$$p_{\theta}(x_{t-1}\mid x_t) = \mathcal{N}\!\left( x_{t-1};\mu_{\theta}(x_t,t), \Sigma_{\theta}(x_t,t) \right).$$

One common parameterization predicts the noise added to $x_0$. A widely used simplified objective is

$$\mathbb{E}_{x_0,t,\epsilon} \left[ \left\| \epsilon-\epsilon_{\theta} \left( \sqrt{\bar{\alpha}_t}x_0 +\sqrt{1-\bar{\alpha}_t}\epsilon, t \right) \right\|_2^2 \right].$$

This noise-prediction loss is connected to the variational objective, but exact weighting and parameterization matter; “predict noise with MSE” is not the complete definition of every diffusion model.

Sampling

1. Draw $x_T$ from the chosen noise distribution.
2. For $t=T,\ldots,1$, use the learned model and sampler to obtain $x_{t-1}$.
3. Return the final $x_0$ representation.

Conditioning supplies additional information—such as a class, text representation, or measurement—to the denoiser. Guidance can strengthen conditioning at the cost of changing the diversity, fidelity, or calibration trade-off.

Design and Evaluation Boundaries

• The noise schedule, prediction target, model architecture, and sampler are separate design choices.
• Standard sampling is iterative and can require many model evaluations; faster samplers trade computation against approximation behavior.
• A low denoising objective does not by itself establish perceptual quality, diversity, likelihood quality, or usefulness.
• Conditional generation can reproduce biases or memorized structure from training data.
• The model's output space, preprocessing, and decoder can be as important as the denoiser.
• Current image, audio, or API products belong in Frontier; this note owns the durable probabilistic process.

The canonical starting point is Denoising Diffusion Probabilistic Models. Framework-specific U-Net and sampling implementations are intentionally left to maintained libraries and papers.