How AI image generators make a picture from your words
Denoising: how an AI image generator makes up a picture out of pure noise
Denoising means removing noise. A trained diffusion model starts from a square of pure noise and removes a little of it at each step. Step by step, a picture appears that never existed before.
-
1
The diffusion model does the one thing it practiced: it guesses the noise, as if a picture lay underneath. Its guess covers nearly the whole square, all but a vague patch where some picture could be.
-
2
A helper program takes out only a little of that guess, not all of it. What is left still looks like noise, but it is no longer pure noise, so the next guess has a bit more to go on.
-
3
Guess, take a little out, guess again: each round is called a step. Large shapes settle first and details come last. Stable Diffusion 1.5 runs 50 steps, unless you pick another number.
-
4
After the last step, your fox sits in the snow. Nothing was ever under the noise: taking it away, step by step, made the picture up. This is called denoising.
-
5
Strange, though: the diffusion model cannot read. Not one letter of your prompt reaches it. So how did this noise turn into a fox with a newspaper, and not a cat on a sofa?
-
Quick check
A shortcut that does not work.
Show the answer
You would get a blur. The first guess was made from pure noise, so all it leaves is a vague patch: small steps let each new guess build on a bit more picture.
-
Go further
The helper and the steps, as engineers name them.