principles.fyi · the brain · concept

autoregression

Write one word, glue it on, run again, repeat.

next_token = model(all_tokens_so_far); then append and repeat

Autoregression means the model writes text one token at a time, where each new token is predicted from everything that came before it — including the tokens it just generated. You take the model's predicted token, append it to the input, and feed the whole thing back in to predict the next one, looping until done. This works because text is sequential: knowing the start strongly narrows down what comes next. Inside a transformer it's the outer loop that turns a next-token predictor into a writer, and it's why generation feels slow — every new token needs its own pass through the model.