Embeddings: meaning as geometry

Token IDs are just labels: ID 512 isn't any closer in meaning to ID 513. So the model's first step is to turn every token into an embedding: a list of hundreds or thousands of numbers, which you can picture as a point in a high-dimensional space.