How AI image generators make a picture from your words
CLIP: how an AI image generator learned what your words look like
An AI image generator does not read your prompt as letters. Before the picture starts, a second AI model, called CLIP, turns your words into numbers. How did CLIP learn what a word like "snow" looks like?
-
1
Stable Diffusion 1.5 is in fact a small team of models, and you meet two of them now. Before the diffusion model starts, another one gets your prompt: the text encoder.
-
2
First it cuts your prompt into pieces it knows, called tokens. Here each of your eight words is one token.
The rest of this episode is for subscribers
Subscribe to read every premium episode. You can cancel when you want.
Paid yearly: $99 instead of $119.88