How AI image generators make a picture from your words
Diffusion models: how an AI image generator learned from pictures buried in noise
An AI image generator must learn before it can make any picture. In training, it gets real photos covered with random noise, again and again. This one strange exercise teaches it to make new pictures.
-
1
Go back to long before you typed. Miva, a researcher, sets a training program to work on the model, with a huge collection of real pictures.
-
2
The training program takes one real picture and pours noise on it, dots it picks at random. It keeps a copy of that noise, so it knows exactly what it added.
-
3
The model gets the noisy picture, but neither the clean picture nor the copy. Its one exercise: guess which dots are the noise.
-
4
The training program lays the guess over its copy and spots where they differ. The model is made of millions of numbers called parameters, and the program turns each one a tiny bit, toward a better guess.
-
5
Then a new picture, with a little noise, or with so much that the picture almost disappears. A few thousand pictures at a time, for hundreds of thousands of rounds. Burying pictures in noise is called diffusion, and a model trained this way is a diffusion model.
-
6
Then training is over. The parameters stop moving for good, and the diffusion model goes out into the world. Its first real job: the square your fox starts from.
-
Quick check
A training program that forgot its copy.
Show the answer
No. Without the copy it cannot see where the guess is wrong, so it does not know which way to turn the parameters.
-
Go further
What engineers call the amount of noise and the size of the miss.