TransformersLesson 1 of 10
One next-token prediction
Token sequences and next-token probabilities
This series follows a decoder language model: it reads a sequence of tokens and predicts a probability distribution for the next token. A token can be a word, part of a word, punctuation, or another text fragment. We use whole words in the small examples to keep them readable.
Given “the cat sat”, it might assign a high probability to on. A decoding rule chooses a token from the scores. Append that token and make the next prediction. The model does not have to select the most probable token every time.
The loop that writes everything
Section titled “The loop that writes everything”Conditioning each prediction on the existing sequence is autoregression. Training the model and using it to generate are different operations: a normal generation step does not update its learned weights.
So “understand a transformer” really means: understand that one guess.
What’s inside the guess
Section titled “What’s inside the guess”Much of the numerical work is matrix multiplication. Its basic calculation is the dot product — multiply two lists of numbers pairwise, add them up, get one number. A matrix multiply is just a whole grid of those, done billions of times.
We mark every matrix multiply in amber, so you can see how much of the machine is really just that.
First, one problem
Section titled “First, one problem”A matrix multiply needs numbers. Our input is words. A computer can’t multiply cat.
First, a tokenizer maps text to token IDs. Each ID selects a learned vector from an embedding table. The next lesson follows that lookup.