NeuralDocs

Advanced Topics / Diffusion Models

Diffusion Models

Advanced

Generative models that learn to reverse a gradual noising process, powering modern image and video generation.

Forward and reverse process

The forward process adds Gaussian noise over T steps. When the cumulative signal coefficient is sufficiently small, the terminal distribution is approximately standard Gaussian. A neural network (usually a U-Net or transformer) learns to predict the noise for reverse sampling.

x₀x_1x_2x_3x_Tforward q: add noisereverse p_θ: denoise
Forward process q (left → right) gradually adds Gaussian noise until x_T is almost pure noise. The learned reverse process p_θ (right → left) removes noise step by step to generate a new sample.
αt=1−βt,αˉt=∏s=1tαsxt=αˉt x0+1−αˉt ϵLsimple=Et,x0,ϵ[∥ϵ−ϵθ(xt,t)∥22]\boldsymbol{\begin{aligned}\alpha_t&=1-\beta_t,\qquad\bar{\alpha}_t=\prod_{s=1}^{t}\alpha_s\\\mathbf{x}_t&=\sqrt{\bar{\alpha}_t}\,\mathbf{x}_0+\sqrt{1-\bar{\alpha}_t}\,\boldsymbol{\epsilon}\\\mathcal{L}_{\mathrm{simple}}&=\mathbb{E}_{t,\mathbf{x}_0,\boldsymbol{\epsilon}}\left[\left\lVert\boldsymbol{\epsilon}-\epsilon_\theta(\mathbf{x}_t,t)\right\rVert_2^2\right]\end{aligned}}

For the simplified DDPM noise-prediction objective, t is uniform over {1, …, T}, x₀ is a data sample, ε ∼ N(0, I), and 0 < βₜ < 1 defines the noise schedule. This is not the full variational objective.

Key ideas

  • DDPM and DDIM sampling
  • Latent diffusion (Stable Diffusion)
  • Classifier-free guidance for text conditioning

Interview questions & answers

  1. Q1. What happens in the forward and reverse diffusion processes?

    Answer: The forward process gradually corrupts data with noise according to a schedule. A trained model guides a reverse sampling process from noise toward a data sample. Reverse generation is learned and stochastic in many samplers; it is not an exact undoing of a specific corruption trajectory.

  2. Q2. Why do many diffusion models learn to predict noise?

    Answer: Known noise can be added to a training sample at a randomly selected time step, providing a direct supervised target. Predicting that noise helps estimate a denoising direction for sampling. Other parameterizations predict the clean sample or a velocity target instead.

  3. Q3. How do latent diffusion and classifier-free guidance help generation?

    Answer: Latent diffusion performs denoising in a compressed representation and decodes the result, reducing compute compared with full-resolution pixel diffusion. Classifier-free guidance combines conditional and unconditional predictions to strengthen prompt adherence, with excessive guidance potentially reducing diversity or causing artifacts.

← Previous

Transfer Learning & Fine-tuning

Next →

Deep Reinforcement Learning