Diffusion model

In machine learning, diffusion models, also known as diffusion-based generative models or score-based generative models, are a class of latent variable generative models.

Uses your system voice; voice and rate in settings.

Snapshot

≈ 30 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In machine learning, diffusion models, also known as diffusion-based generative models or score-based generative models, are a class of latent variable generative models. A diffusion model consists of two major components: the forward diffusion process, and the reverse sampling process.

The goal of diffusion models is to learn a diffusion process for a given dataset, such that the process can generate new elements that are distributed similarly as the original dataset. A diffusion model models data as generated by a diffusion process, whereby a new datum performs a random walk with drift through the space of all possible data.

A trained diffusion model can be sampled in many ways, with different efficiency and quality. There are various equivalent formalisms, including Markov chains, denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. They are typically trained using variational inference.

The model responsible for denoising is typically called its "backbone". The backbone may be of any kind, but they are typically U-nets or transformers. , diffusion models are mainly used for computer vision tasks, including image denoising, inpainting, super-resolution, image generation, and video generation.

These typically involve training a neural network to sequentially denoise images blurred with Gaussian noise.…

In machine learning, diffusion models, also known as diffusion-based generative models or score-based generative models, are a class of latent variable generative models. A diffusion model consists of two major components: the forward diffusion process, and the reverse sampling process.

The goal of diffusion models is to learn a diffusion process for a given dataset, such that the process can generate new elements that are distributed similarly as the original dataset. A diffusion model models data as generated by a diffusion process, whereby a new datum performs a random walk with drift through the space of all possible data.

A trained diffusion model can be sampled in many ways, with different efficiency and quality. There are various equivalent formalisms, including Markov chains, denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations. They are typically trained using variational inference.

The model responsible for denoising is typically called its "backbone". The backbone may be of any kind, but they are typically U-nets or transformers. , diffusion models are mainly used for computer vision tasks, including image denoising, inpainting, super-resolution, image generation, and video generation.

These typically involve training a neural network to sequentially denoise images blurred with Gaussian noise. The model is trained to reverse the process of adding noise to an image. After training to convergence, it can be used for image generation by starting with an image composed of random noise, and applying the network iteratively to denoise the image.

Diffusion-based image generators have seen widespread commercial interest, such as Stable Diffusion and DALL-E. These models typically combine diffusion models with other models, such as text-encoders and cross-attention modules to allow text-conditioned generation.

Other than computer vision, diffusion models have also found applications in natural language processing such as text generation and summarization, sound generation, and reinforcement learning. The equilibrium distribution is the Gaussian distribution \mathcal{N}(0, I) , with pdf \rho(x) \propto e^{-\frac 12 \|x\|^2} .

This is just the Maxwell–Boltzmann distribution of particles in a potential well V(x) = \frac 12 \|x\|^2 at temperature 1. The initial distribution, being very much out of equilibrium, would diffuse towards the equilibrium distribution, making biased random steps that are a sum of pure randomness (like a Brownian walker) and gradient descent down the potential well.

The randomness is necessary: if the particles were to undergo only gradient descent, then they will all fall to the origin, collapsing the distribution. \alpha_t := 1-\beta_t \bar \alpha_t := \alpha_1 \cdots \alpha_t \sigma_t := \sqrt{1 -\bar \alpha_t} \tilde \sigma_t := \frac{\sigma_{t-1}}{\sigma_{t}}\sqrt{\beta_t} \tilde\mu_t(x_t, x_0) :=\frac{\sqrt{\alpha_{t}}(1-\bar \alpha_{t-1})x_t +\sqrt{\bar\alpha_{t-1}}(1-\alpha_{t})x_0}{\sigma_{t}^2} \mathcal{N}(\mu, \Sigma) is the Gaussian distribution with mean \mu and variance \Sigma , and \mathcal{N}(x | \mu, \Sigma) is the probability density at x . A vertical bar denotes conditioning. A forward diffusion process starts at some starting point x_0 \sim q , where q is the probability distribution to be learned, then repeatedly adds noise to it by x_t = \sqrt{1-\beta_t} x_{t-1} + \sqrt{\beta_t} z_t where z_1, ..., z_T are IID (Independent and identically distributed random variables) samples from \mathcal{N}(0, I) . The coefficients \sqrt{1 - \beta_t} and \sqrt{\beta_t} ensure that \mbox{Var}(X_t) = I assuming that \mbox{Var}(X_0) = I . The values of \beta_t are chosen such that for any starting distribution of x_0 , if it has finite second moment, then \lim_{t\to\infty} x_t|x_0 converges to \mathcal{N}(0, I) . The entire diffusion process then satisfies q(x_{0:T}) = q(x_0)q(x_1|x_0) \cdots q(x_T|x_{T-1}) = q(x_0) \mathcal{N}(x_1 | \sqrt{\alpha_1} x_0, \beta_1 I) \cdots \mathcal{N}(x_T | \sqrt{\alpha_T} x_{T-1}, \beta_T I) or \ln q(x_{0:T}) = \ln q(x_0) - \sum_{t=1}^T \frac{1}{2\beta_t} \| x_t - \sqrt{1-\beta_t}x_{t-1}\|^2 + C where C is a normalization constant and often omitted.

In particular, we note that x_{1:T}|x_0 is a Gaussian process, which affords us considerable freedom in reparameterization. For example, by standard manipulation with Gaussian process, x_{t}|x_0 \sim N\left(\sqrt{\bar\alpha_t} x_{0}, \sigma_{t}^2 I \right) x_{t-1} | x_t, x_0 \sim \mathcal{N}(\tilde\mu_t(x_t, x_0), \tilde\sigma_t^2 I) In particular, notice that for large t , the variable x_{t}|x_0 \sim N\left(\sqrt{\bar\alpha_t} x_{0}, \sigma_{t}^2 I \right) converges to \mathcal{N}(0, I) . That is, after a long enough diffusion process, we end up with some x_T that is very close to \mathcal{N}(0, I) , with all traces of the original x_0 \sim q gone.

For example, since x_{t}|x_0 \sim N\left(\sqrt{\bar\alpha_t} x_{0}, \sigma_{t}^2 I \right) we can sample x_{t}|x_0 directly "in one step", instead of going through all the intermediate steps x_1, x_2, ..., x_{t-1} . {{Math proof|title=Derivation by reparameterization|proof= We know x_{t-1}|x_0 is a Gaussian, and x_t|x_{t-1} is another Gaussian. We also know that these are independent.

Thus we can perform a reparameterization: x_{t-1} = \sqrt{\bar\alpha_{t-1}} x_{0} + \sqrt{1 - \bar\alpha_{t-1}} z x_t = \sqrt{\alpha_t} x_{t-1} + \sqrt{1-\alpha_t} z' where z, z' are IID Gaussians. There are 5 variables x_0, x_{t-1}, x_t, z, z' and two linear equations.

The two sources of randomness are z, z' , which can be reparameterized by rotation, since the IID Gaussian distribution is rotationally symmetric. By plugging in the equations, we can solve for the first reparameterization: x_t = \sqrt{\bar \alpha_t}x_0 + \underbrace{\sqrt{\alpha_t - \bar\alpha_t}z + \sqrt{1-\alpha_t}z'}_{= \sigma_t z} where z is a Gaussian with mean zero and variance one.

To find the second one, we complete the rotational matrix: \begin{bmatrix}z \\z\end{bmatrix} = \begin{bmatrix} \frac{\sqrt{\alpha_t - \bar\alpha_t}}{\sigma_t} & \frac{\sqrt{\beta_t}}{\sigma_t} \\?&?\end{bmatrix} \begin{bmatrix} z\\z'\end{bmatrix} Since rotational matrices are all of the form \begin{bmatrix} \cos\theta & \sin\theta\\ -\sin\theta & \cos\theta \end{bmatrix} , we know the matrix must be \begin{bmatrix}z \\z\end{bmatrix} = \begin{bmatrix} \frac{\sqrt{\alpha_t - \bar\alpha_t}}{\sigma_t} & \frac{\sqrt{\beta_t}}{\sigma_t} \\- \frac{\sqrt{\beta_t}}{\sigma_t} & \frac{\sqrt{\alpha_t - \bar\alpha_t}}{\sigma_t} \end{bmatrix} \begin{bmatrix} z\\z'\end{bmatrix} and since the inverse of rotational matrix is its transpose, \begin{bmatrix}z \\z'\end{bmatrix} = \begin{bmatrix} \frac{\sqrt{\alpha_t - \bar\alpha_t}}{\sigma_t} & -\frac{\sqrt{\beta_t}}{\sigma_t} \\ \frac{\sqrt{\beta_t}}{\sigma_t} & \frac{\sqrt{\alpha_t - \bar\alpha_t}}{\sigma_t} \end{bmatrix} \begin{bmatrix} z\\z\end{bmatrix} Plugging back, and simplifying, we have x_t = \sqrt{\bar\alpha_t}x_0 + \sigma_tz x_{t-1} = \tilde\mu_t(x_t, x_0) - \tilde\sigma_t z }} The key idea of DDPM is to use a neural network parametrized by \theta . The network takes in two arguments x_t, t , and outputs a vector \mu_\theta(x_t, t) and a matrix \Sigma_\theta(x_t, t) , such that each step in the forward diffusion process can be approximately undone by x_{t-1} \sim \mathcal{N}(\mu_\theta(x_t, t), \Sigma_\theta(x_t, t)) . This then gives us a backward diffusion process p_\theta defined by p_\theta(x_T) = \mathcal{N}(x_T | 0, I) p_\theta(x_{t-1} | x_t) = \mathcal{N}(x_{t-1} | \mu_\theta(x_t, t), \Sigma_\theta(x_t, t)) The goal now is to learn the parameters \theta such that p_\theta(x_0) is as close to q(x_0) as possible.

To do that, we use maximum likelihood estimation with variational inference. The ELBO inequality states that \ln p_\theta(x_0) \geq E_{x_{1:T}\sim q(\cdot | x_0)}[ \ln p_\theta(x_{0:T}) - \ln q(x_{1:T}|x_0)] , and taking one more expectation, we get E_{x_0 \sim q}[\ln p_\theta(x_0)] \geq E_{x_{0:T}\sim q}[ \ln p_\theta(x_{0:T}) - \ln q(x_{1:T}|x_0)] We see that maximizing the quantity on the right would give us a lower bound on the likelihood of observed data.

This allows us to perform variational inference. Define the loss function L(\theta) := -E_{x_{0:T}\sim q}[ \ln p_\theta(x_{0:T}) - \ln q(x_{1:T}|x_0)] and now the goal is to minimize the loss by stochastic gradient descent.

The expression may be simplified to L(\theta) = \sum_{t=1}^T E_{x_{t-1}, x_t\sim q}[-\ln p_\theta(x_{t-1} | x_t)] + E_{x_0 \sim q}[D_{KL}(q(x_T|x_0) \| p_\theta(x_T))] + C where C does not depend on the parameter, and thus can be ignored. Since p_\theta(x_T) = \mathcal{N}(x_T | 0, I) also does not depend on the parameter, the term E_{x_0 \sim q}[D_{KL}(q(x_T|x_0) \| p_\theta(x_T))] can also be ignored.

This leaves just L(\theta ) = \sum_{t=1}^T L_t with L_t = E_{x_{t-1}, x_t\sim q}[-\ln p_\theta(x_{t-1} | x_t)] to be minimized. Since x_{t-1} | x_t, x_0 \sim \mathcal{N}(\tilde\mu_t(x_t, x_0), \tilde\sigma_t^2 I) , this suggests that we should use \mu_\theta(x_t, t) = \tilde \mu_t(x_t, x_0) ; however, the network does not have access to x_0 , and so it has to estimate it instead.

Now, since x_{t}|x_0 \sim N\left(\sqrt{\bar\alpha_t} x_{0}, \sigma_{t}^2 I \right) , we may write x_t = \sqrt{\bar\alpha_t} x_{0} + \sigma_t z , where z is some unknown Gaussian noise. Now we see that estimating x_0 is equivalent to estimating z . Therefore, let the network output a noise vector \epsilon_\theta(x_t, t) , and let it predict \mu_\theta(x_t, t) =\tilde\mu_t\left(x_t, \frac{x_t - \sigma_t \epsilon_\theta(x_t, t)}{\sqrt{\bar\alpha_t}}\right) = \frac{x_t - \epsilon_\theta(x_t, t) \beta_t/\sigma_t}{\sqrt{\alpha_t}} It remains to design \Sigma_\theta(x_t, t) . The DDPM paper suggested not learning it (since it resulted in "unstable training and poorer sample quality"), but fixing it at some value \Sigma_\theta(x_t, t) = \zeta_t^2 I , where either \zeta_t^2 = \beta_t \text{ or } \tilde\sigma_t^2 yielded similar performance.

With this, the loss simplifies to L_t = \frac{\beta_t^2}{2\alpha_t\sigma_{t}^2\zeta_t^2} E_{x_0\sim q; z \sim \mathcal{N}(0, I)}\left[ \left\| \epsilon_\theta(x_t, t) - z \right\|^2\right] + C which may be minimized by stochastic gradient descent. The paper noted empirically that an even simpler loss function L_{simple, t} = E_{x_0\sim q; z \sim \mathcal{N}(0, I)}\left[ \left\| \epsilon_\theta(x_t, t) - z \right\|^2\right] resulted in better models.

One, we no longer need to normalize q(x) , but can use any \tilde q(x) = Cq(x) , where C = \int \tilde q(x) dx > 0 is any unknown constant that is of no concern to us. Two, we are comparing q(x) neighbors q(x + dx) , by \frac{q(x)}{q(x+dx)} =e^{-\langle \nabla_x \ln q, dx \rangle} Let the score function be s(x) := \nabla_x \ln q(x) ; then consider what we can do with s(x) . As it turns out, s(x) allows us to sample from q(x) using thermodynamics. Specifically, if we have a potential energy function U(x) = -\ln q(x) , and a lot of particles in the potential well, then the distribution at thermodynamic equilibrium is the Boltzmann distribution q_U(x) \propto e^{-U(x)/k_B T} = q(x)^{1/k_BT} .

At temperature k_BT=1 , the Boltzmann distribution is exactly q(x) . Therefore, to model q(x) , we may start with a particle sampled at any convenient distribution (such as the standard Gaussian distribution), then simulate the motion of the particle forwards according to the Langevin equation dx_{t}= -\nabla_{x_t}U(x_t) d t+d W_t and the Boltzmann distribution is, by Fokker-Planck equation, the unique thermodynamic equilibrium. So no matter what distribution x_0 has, the distribution of x_t converges in distribution to q as t\to \infty .

Given a density q , we wish to learn a score function approximation f_\theta \approx \nabla \ln q . This is score matching. Typically, score matching is formalized as minimizing Fisher divergence function E_q[\|f_\theta(x) - \nabla \ln q(x)\|^2] .

By expanding the integral, and performing an integration by parts, E_q[\|f_\theta(x) - \nabla \ln q(x)\|^2] = E_q[\|f_\theta\|^2 + 2\nabla\cdot f_\theta] + C giving us a loss function, also known as the Hyvärinen scoring rule, that can be minimized by stochastic gradient descent. Suppose we need to model the distribution of images, and we want x_0 \sim \mathcal{N}(0, I) , a white-noise image.

Now, most white-noise images do not look like real images, so q(x_0) \approx 0 for large swaths of x_0 \sim \mathcal{N}(0, I) . This presents a problem for learning the score function, because if there are no samples around a certain point, then we can't learn the score function at that point. If we do not know the score function \nabla_{x_t}\ln q(x_t) at that point, then we cannot impose the time-evolution equation on a particle: dx_{t}= \nabla_{x_t}\ln q(x_t) d t+d W_t To deal with this problem, we perform annealing.

If q is too different from a white-noise distribution, then progressively add noise until it is indistinguishable from one. That is, we perform a forward diffusion, then learn the score function, then use the score function to perform a backward diffusion.

Consider again the forward diffusion process, but this time in continuous time: x_t = \sqrt{1-\beta_t} x_{t-1} + \sqrt{\beta_t} z_t By taking the \beta_t \to \beta(t)dt, \sqrt{dt}z_t \to dW_t limit, we obtain a continuous diffusion process, in the form of a stochastic differential equation: dx_t = -\frac 12 \beta(t) x_t dt + \sqrt{\beta(t)} dW_t where W_t is a Wiener process (multidimensional Brownian motion). Now, the equation is exactly a special case of the overdamped Langevin equation dx_t = -\frac{D}{k_BT} (\nabla_x U)dt + \sqrt{2D}dW_t where D is diffusion tensor, T is temperature, and U is potential energy field.

If we substitute in D= \frac 12 \beta(t)I, k_BT = 1, U = \frac 12 \|x\|^2 , we recover the above equation. This explains why the phrase "Langevin dynamics" is sometimes used in diffusion models. Now the above equation is for the stochastic motion of a single particle.

Suppose we have a cloud of particles distributed according to q at time t=0 , then after a long time, the cloud of particles would settle into the stable distribution of \mathcal{N}(0, I) . Let \rho_t be the density of the cloud of particles at time t , then we have \rho_0 = q; \quad \rho_T \approx \mathcal{N}(0, I) and the goal is to somehow reverse the process, so that we can start at the end and diffuse back to the beginning. By Fokker-Planck equation, the density of the cloud evolves according to \partial_t \ln \rho_t = \frac 12 \beta(t) \left( n + (x+ \nabla\ln\rho_t) \cdot \nabla \ln\rho_t + \Delta\ln\rho_t \right) where n is the dimension of space, and \Delta is the Laplace operator.

Equivalently, \partial_t \rho_t = \frac 12 \beta(t) ( \nabla\cdot(x\rho_t) + \Delta \rho_t) If we have solved \rho_t for time t\in [0, T] , then we can exactly reverse the evolution of the cloud. Suppose we start with another cloud of particles with density \nu_0 = \rho_T , and let the particles in the cloud evolve according to dy_t = \frac{1}{2} \beta(T-t) y_{t} d t + \beta(T-t) \underbrace{\nabla_{y_{t}} \ln \rho_{T-t}\left(y_{t}\right)}_{\text {score function }} d t+\sqrt{\beta(T-t)} d W_t then by plugging into the Fokker-Planck equation, we find that \partial_t \rho_{T-t} = \partial_t \nu_t . Thus this cloud of points is the original cloud, evolving backwards. \bar \alpha_t = (1-\beta_1) \cdots (1-\beta_t) = e^{\sum_i \ln(1-\beta_i)} \to e^{-\int_0^t \beta(t)dt} and so x_{t}|x_0 \sim N\left(e^{-\frac 12\int_0^t \beta(t)dt} x_{0}, \left(1- e^{-\int_0^t \beta(t)dt}\right) I \right) In particular, we see that we can directly sample from any point in the continuous diffusion process without going through the intermediate steps, by first sampling x_0 \sim q, z \sim \mathcal{N}(0, I) , then get x_t = e^{-\frac 12\int_0^t \beta(t)dt} x_{0} + \left(1- e^{-\int_0^t \beta(t)dt}\right) z . That is, we can quickly sample x_t \sim \rho_t for any t \geq 0 .

Now, define a certain probability distribution \gamma over [0, \infty) , then the score-matching loss function is defined as the expected Fisher divergence: L(\theta) = E_{t\sim \gamma, x_t \sim \rho_t}[\|f_\theta(x_t, t)\|^2 + 2\nabla\cdot f_\theta(x_t, t)] After training, f_\theta(x_t, t) \approx \nabla \ln\rho_t , so we can perform the backwards diffusion process by first sampling x_T \sim \mathcal{N}(0, I) , then integrating the SDE from t=T to t=0 : x_{t-dt}=x_t + \frac{1}{2} \beta(t) x_{t} d t + \beta(t) f_\theta(x_t, t) d t+\sqrt{\beta(t)} d W_t This may be done by any SDE integration method, such as Euler–Maruyama method. The name "noise conditional score network" is explained thus: "network", because f_\theta is implemented as a neural network.

"score", because the output of the network is interpreted as approximating the score function \nabla\ln\rho_t . "noise conditional", because \rho_t is equal to \rho_0 blurred by an added Gaussian noise that increases with time, and so the score function depends on the amount of noise added. \nabla_{x_t}\ln q(x_t) = \frac{1}{\sigma_{t}^2}(-x_t + \sqrt{\bar\alpha_t} E_q[x_0|x_t]) As described previously, the DDPM loss function is \sum_t L_{simple, t} with L_{simple, t} = E_{x_0\sim q; z \sim \mathcal{N}(0, I)}\left[ \left\| \epsilon_\theta(x_t, t) - z \right\|^2\right] where x_t =\sqrt{\bar\alpha_t} x_{0} + \sigma_tz . By a change of variables, L_{simple, t} = E_{x_0, x_t\sim q}\left[ \left\| \epsilon_\theta(x_t, t) - \frac{x_t -\sqrt{\bar\alpha_t} x_{0}}{\sigma_t} \right\|^2\right] = E_{x_t\sim q, x_0\sim q(\cdot | x_t)}\left[ \left\| \epsilon_\theta(x_t, t) - \frac{x_t -\sqrt{\bar\alpha_t} x_{0}}{\sigma_t} \right\|^2\right] and the term inside becomes a least squares regression, so if the network actually reaches the global minimum of loss, then we have \epsilon_\theta(x_t, t) = \frac{x_t -\sqrt{\bar\alpha_t} E_q[x_0|x_t]}{\sigma_t} = -\sigma_t\nabla_{x_t}\ln q(x_t) Thus, given a good score-based network, its predicted score is a good prediction of the noise (after scaling by \sigma_t ), and thus can be used for denoising.

Conversely, the continuous limit x_{t-1} = x_{t-dt}, \beta_t = \beta(t) dt, z_t\sqrt{dt} = dW_t of the backward equation x_{t-1} = \frac{x_t}{\sqrt{\alpha_t}}- \frac{ \beta_t}{\sigma_{t}\sqrt{\alpha_t }} \epsilon_\theta(x_t, t) + \sqrt{\beta_t} z_t; \quad z_t \sim \mathcal{N}(0, I) gives us precisely the same equation as score-based diffusion: x_{t-dt} = x_t(1+\beta(t)dt / 2) + \beta(t) \nabla_{x_t}\ln q(x_t) dt + \sqrt{\beta(t)}dW_t Thus, at infinitesimal steps of DDPM, a denoising network performs score-based diffusion. In DDPM, the sequence of numbers 0 = \sigma_0 is called a (discrete time) noise schedule.

In general, consider a strictly increasing monotonic function \sigma of type \R \to (0, 1) , such as the sigmoid function. In that case, a noise schedule is a sequence of real numbers \lambda_1 . It then defines a sequence of noises \sigma_t := \sigma(\lambda_t) , which then derives the other quantities \beta_t = 1 - \frac{1 - \sigma_t^2}{1 - \sigma_{t-1}^2} .

In order to use arbitrary noise schedules, instead of training a noise prediction model \epsilon_\theta(x_t, t) , one trains \epsilon_\theta(x_t, \sigma_t) . Similarly, for the noise conditional score network, instead of training f_\theta(x_t, t) , one trains f_\theta(x_t, \sigma_t) . In detail, the DDIM sampling method is as follows. Start with the forward diffusion process x_t = \sqrt{\bar\alpha_t} x_0 + \sigma_t \epsilon .

Then, during the backward denoising process, given x_t, \epsilon_\theta(x_t, t) , the original data is estimated as x_0' = \frac{x_t - \sigma_t \epsilon_\theta(x_t, t)}{ \sqrt{\bar\alpha_t}} then the backward diffusion process can jump to any step 0 \leq s , and the next denoised sample is x_{s} = \sqrt{\bar\alpha_{s}} x_0' + \sqrt{\sigma_{s}^2 - (\sigma'_s)^2} \epsilon_\theta(x_t, t) + \sigma_s' \epsilon where \sigma_s' is an arbitrary real number within the range [0, \sigma_s] , and \epsilon \sim \mathcal{N}(0, I) is a newly sampled Gaussian noise. If all \sigma_s' = 0 , then the backward process becomes deterministic, and this special case of DDIM is also called "DDIM".

The original paper noted that when the process is deterministic, samples generated with only 20 steps are already very similar to ones generated with 1000 steps on the high-level. The original paper recommended defining a single "eta value" \eta \in [0, 1] , such that \sigma_s' = \eta \tilde\sigma_s . When \eta = 1 , this is the original DDPM.

When \eta = 0 , this is the fully deterministic DDIM. For intermediate values, the process interpolates between them. By the equivalence, the DDIM algorithm also applies for score-based diffusion models.

In the v-prediction formalism, the noising formula x_t = \sqrt{\bar\alpha_t} x_0 + \sqrt{1 - \bar\alpha_t} \epsilon_t is reparameterised by an angle \phi_t such that \cos \phi_t = \sqrt{\bar\alpha_t} and a "velocity" defined by \cos\phi_t \epsilon_t - \sin\phi_t x_0 . The network is trained to predict the velocity \hat v_\theta , and denoising is by x_{\phi_t - \delta} = \cos(\delta)\; x_{\phi_t} - \sin(\delta) \hat{v}_{\theta}\; (x_{\phi_t}) .

This parameterization was found to improve performance, as the model can be trained to reach total noise (i.e. \phi_t = 90^\circ ) and then reverse it, whereas the standard parameterization never reaches total noise since \sqrt{\bar\alpha_t} > 0 is always true. p(x|y) \propto p(y|x)p(x) in other words, if we have a good model of the space of all images, and a good image-to-class translator, we get a class-to-image translator "for free". In the equation for backward diffusion, the score \nabla \ln p(x) can be replaced by \nabla_x \ln p(x|y) = \underbrace{\nabla_x \ln p(x)}_{\text{score}} + \underbrace{\nabla_x \ln p(y|x)}_{\text{classifier guidance}} where \nabla_x \ln p(x) is the score function, trained as previously described, and \nabla_x \ln p(y|x) is found by using a differentiable image classifier.

During the diffusion process, we need to condition on the time, giving \nabla_{x_t} \ln p(x_t|y, t) = \nabla_{x_t} \ln p(y|x_t, t) + \nabla_{x_t} \ln p(x_t|t) Although, usually the classifier model does not depend on time, in which case p(y|x_t, t) = p(y|x_t) . Classifier guidance is defined for the gradient of score function, thus for score-based diffusion network, but as previously noted, score-based diffusion models are equivalent to denoising models by \epsilon_\theta(x_t, t) = -\sigma_t\nabla_{x_t}\ln p(x_t|t) , and similarly, \epsilon_\theta(x_t, y, t) = -\sigma_t\nabla_{x_t}\ln p(x_t|y, t) . Therefore, classifier guidance works for denoising diffusion as well, using the modified noise prediction: \epsilon_\theta(x_t, y, t) = \epsilon_\theta(x_t, t) - \underbrace{\sigma_t \nabla_{x_t} \ln p(y|x_t, t)}_{\text{classifier guidance}} p_\gamma(x|y) \propto p(y|x)^\gamma p(x) where \gamma > 0 is interpretable as inverse temperature. In the context of diffusion models, it is usually called the guidance scale.

A high \gamma would force the model to sample from a distribution concentrated around \arg\max_x p(y|x) . This sometimes improves quality of generated images. This gives a modification to the previous equation: \nabla_x \ln p_\beta(x|y) = \nabla_x \ln p(x) + \gamma \nabla_x \ln p(y|x) For denoising models, it corresponds to \epsilon_\theta(x_t, y, t) = \epsilon_\theta(x_t, t) - \gamma \sigma_t \nabla_{x_t} \ln p(y|x_t, t) \nabla_x \ln p_\gamma(x|y) = (1-\gamma) \nabla_x \ln p(x) + \gamma \nabla_x \ln p(x|y) Such a model is usually trained by presenting it with both (x, y) and (x, {\rm None}) , allowing it to model both \nabla_x\ln p(x|y) and \nabla_x\ln p(x) . Note that for CFG, the diffusion model cannot be merely a generative model of the entire data distribution \nabla_x \ln p(x) . It must be a conditional generative model \nabla_x \ln p(x | y) . For example, in stable diffusion, the diffusion backbone takes as input both a noisy model x_t , a time t , and a conditioning vector y (such as a vector encoding a text prompt), and produces a noise prediction \epsilon_\theta(x_t, y, t) . For denoising models, it corresponds to \epsilon_\theta(x_t, y, t, \gamma) = \epsilon_\theta(x_t, t) + \gamma (\epsilon_\theta(x_t, y, t) - \epsilon_\theta(x_t, t)) As sampled by DDIM, the algorithm can be written as \begin{aligned} \epsilon_{\text{uncond}} &\leftarrow \epsilon_\theta(x_t, t) \\ \epsilon_{\text{cond}} &\leftarrow \epsilon_\theta(x_t, t, c) \\ \epsilon_{\text{CFG}} &\leftarrow \epsilon_{\text{uncond}} + \gamma(\epsilon_{\text{cond}} - \epsilon_{\text{uncond}})\\ x_0 &\leftarrow (x_t - \sigma_t \epsilon_{\text{CFG}}) / \sqrt{1 - \sigma_t^2}\\ x_s &\leftarrow \sqrt{1 - \sigma_s^2} x_0 + \sqrt{\sigma_s^2 - (\sigma_s')^2} \epsilon_{\text{uncond}} + \sigma_s' \epsilon\\ \end{aligned} A similar technique applies to language model sampling.

Also, if the unconditional generation \epsilon_{\text{uncond}} \leftarrow \epsilon_\theta(x_t, t) is replaced by \epsilon_{\text{neg cond}} \leftarrow \epsilon_\theta(x_t, t, c') , then it results in negative prompting, which pushes the generation away from c' condition. Given a diffusion model, one may regard it either as a continuous process, and sample from it by integrating a SDE, or one can regard it as a discrete process, and sample from it by iterating the discrete steps.

The choice of the "noise schedule" \beta_t can also affect the quality of samples. A noise schedule is a function that sends a natural number to a noise level: t \mapsto \beta_t, \quad t \in \{1, 2, \dots\}, \beta \in (0, 1) A noise schedule is more often specified by a map t \mapsto \sigma_t . The two definitions are equivalent, since \beta_t = 1 - \frac{1 - \sigma_t^2}{1 - \sigma_{t-1}^2} .

In the DDPM perspective, one can use the DDPM itself (with noise), or DDIM (with adjustable amount of noise). The case where one adds noise is sometimes called ancestral sampling. One can interpolate between noise and no noise.

The amount of noise is denoted \eta ("eta value") in the DDIM paper, with \eta = 0 denoting no noise (as in deterministic DDIM), and \eta = 1 denoting full noise (as in DDPM). In the perspective of SDE, one can use any of the numerical integration methods, such as Euler–Maruyama method, Heun's method, linear multistep methods, etc. Just as in the discrete case, one can add an adjustable amount of noise during the integration.

A survey and comparison of samplers in the context of image generation is in. \frac{d}{dt} \phi_t(x) = v_t(\phi_t(x)) \quad t \in [0,1], \quad \text{starting from }\phi_0(x) = x we end up with a point x_1 \sim \pi_1 . The solution \phi_t of the above ODE define a probability path p_t = [\phi_t]_{\#} \pi_0 by the pushforward measure operator. In particular, [\phi_1]_{\#} \pi_0 = \pi_1 .

The probability path and the velocity field also satisfy the continuity equation, in the sense of probability distribution: \partial_t p_t + \nabla \cdot (v_t p_t) = 0 To construct a probability path, we start by construct a conditional probability path p_t(x \vert z) and the corresponding conditional velocity field v_t(x \vert z) on some conditional distribution q(z) . A natural choice is the Gaussian conditional probability path: p_t(x \vert z) = \mathcal{N} \left( m_t(z), \zeta_t^2 I \right) The conditional velocity field which corresponds to the geodesic path between conditional Gaussian path is v_t(x \vert z) = \frac{\zeta_t'}{\zeta_t} (x - m_t(z)) + m_t'(z) The probability path and velocity field are then computed by marginalizing p_t(x) = \int p_t(x \vert z) q(z) dz \qquad \text{ and } \qquad v_t(x) = \mathbb{E}_{q(z)} \left[\frac{v_t(x \vert z) p_t(x \vert z)}{p_t(x)} \right] : z = (x_0, x_1) and q(z) = \Gamma(\pi_0, \pi_1) , where \Gamma is the optimal transport plan, which can be approximated by mini-batch optimal transport. If the batch size is not large, then the transport it computes can be very far from the true optimal transport.

The idea of rectified flow is to learn a flow model such that the velocity is nearly constant along each flow path. This is beneficial, because we can integrate along such a vector field with very few steps.

For example, if an ODE \dot{\phi_t}(x) = v_t(\phi_t(x)) follows perfectly straight paths, it simplifies to \phi_t(x) = x_0 + t \cdot v_0(x_0) , allowing for exact solutions in one step. In practice, we cannot reach such perfection, but when the flow field is nearly so, we can take a few large steps instead of many little steps.

The general idea is to start with two distributions \pi_0 and \pi_1 , then construct a flow field \phi^0 = \{\phi_t: t\in[0,1]\} from it, then repeatedly apply a "reflow" operation to obtain successive flow fields \phi^1, \phi^2, \dots , each straighter than the previous one. When the flow field is straight enough for the application, we stop.

Generally, for any time-differentiable process \phi_t , v_t can be estimated by solving: \min_{\theta} \int_0^1 \mathbb{E}_{x \sim p_t}\left [\lVert{v_t(x, \theta) - v_t(x)}\rVert^2\right] \,\mathrm{d}t. In rectified flow, by injecting strong priors that intermediate trajectories are straight, it can achieve both theoretical relevance for optimal transport and computational efficiency, as ODEs with straight paths can be simulated precisely without time discretization. Specifically, rectified flow seeks to match an ODE with the marginal distributions of the linear interpolation between points from distributions \pi_0 and \pi_1 .

Given observations x_0 \sim \pi_0 and x_1 \sim \pi_1 , the canonical linear interpolation x_t= t x_1 + (1-t)x_0, t\in [0,1] yields a trivial case \dot{x}_t = x_1 - x_0 , which cannot be causally simulated without x_1 . To address this, x_t is "projected" into a space of causally simulatable ODEs, by minimizing the least squares loss with respect to the direction x_1 - x_0 : \min_{\theta} \int_0^1 \mathbb{E}_{\pi_0, \pi_1, p_t}\left [\lVert{(x_1-x_0) - v_t(x_t)}\rVert^2\right] \,\mathrm{d}t. The data pair (x_0, x_1) can be any coupling of \pi_0 and \pi_1 , typically independent (i.e., (x_0,x_1) \sim \pi_0 \times \pi_1 ) obtained by randomly combining observations from \pi_0 and \pi_1 .

This process ensures that the trajectories closely mirror the density map of x_t trajectories but reroute at intersections to ensure causality. A distinctive aspect of rectified flow is its capability for "reflow", which straightens the trajectory of ODE paths. Denote the rectified flow \phi^0 = \{\phi_t: t\in[0,1]\} induced from (x_0,x_1) as \phi^0 = \mathsf{Rectflow}((x_0,x_1)) .

Recursively applying this \mathsf{Rectflow}(\cdot) operator generates a series of rectified flows \phi^{k+1} = \mathsf{Rectflow}((\phi_0^k(x_0), \phi_1^k(x_1))) . This "reflow" process not only reduces transport costs but also straightens the paths of rectified flows, making \phi^k paths straighter with increasing k . Rectified flow includes a nonlinear extension where linear interpolation x_t is replaced with any time-differentiable curve that connects x_0 and x_1 , given by x_t = \alpha_t x_1 + \beta_t x_0 .

This framework encompasses DDIM and probability flow ODEs as special cases, with particular choices of \alpha_t and \beta_t . However, in the case where the path of x_t is not straight, the reflow process no longer ensures a reduction in convex transport costs, and also no longer straighten the paths of \phi_t . These linear flow formulations were independently discovered and adapted from a different perspective for supervised inverse problems. For example, Inversion by Direct Iteration (InDI) formulates image restoration by learning a residual flow ODE that iteratively reverses a linear interpolation between a degraded input and a high-quality target to avoid regression-to-the-mean.

InDI is effective for a variety of image restoration tasks. x_{t_x} &= \sqrt{\bar\alpha_{t_x}} x_0 + \sigma_{t_x} \epsilon_x\\ z_{t_z} &= \sqrt{\bar\alpha_{t_z}} z_0 + \sigma_{t_z} \epsilon_z \end{cases} . Train the denoising network to predict \epsilon_x given x_{t_x}, z_{t_z}, t_x, t_z, c . That is, apply gradient descent on \theta on the L2 loss \| \epsilon_\theta(x_{t_x}, z_{t_z}, t_x, t_z, c) - \epsilon_x \|_2^2 .

See also

Diffusion process Markov chain Variational inference Variational autoencoder

References

__FORCETOC__

Text derived from https://en.wikipedia.org/w/index.php?oldid=1375484555 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

35 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.