How ChatGPT and other AI chatbots answer your question
Embeddings: how a list of numbers can hold the meaning of a word
For an AI model, a token is only an ID number, like a line number in a long list. This number says nothing about what the word means. So how does the model know that "cat" and "dog" are close in meaning?
-
1
The model has a fix for that: a huge table, with one row for each token of the vocabulary. It uses a token ID for one thing only: to find that token's row.
-
2
That row holds a long list of numbers, and from now on, it stands for your "plant". Such a list is a vector, and a token's vector is its embedding.
-
3
Tokens that turn up in the same kind of sentences get close embeddings. "plant" and "leaves" hold numbers near each other, and "invoice" holds very different ones.
-
4
Rono typed none of these numbers, and nor did anyone else. They were worked out from huge amounts of text, before your question ever came.
-
5
So inside the model, your question is only numbers, with no book or web page to look in. Where does "Check the soil first" come from? And why can the same machine, in the very same chat, name a book that was never written?
-
Quick check
One word, two meanings.
Show the answer
No. One token, one row: both get the same embedding from the table.
-
Go further
How long an embedding is, and the real neighbours of "plant".