principles.fyi · the brain · concept

context window

How much text the model can read and use at once.

compute to relate all tokens grows like n^2 for n tokens

The context window is the maximum number of tokens (word-pieces) the model can take in at one time — say 8k or 128k of them. It works because the model's attention lets every token look at every other token in the window to decide what matters, so a bigger window means more text it can weigh together. The catch: this all-to-all comparison grows with the square of the length, so doubling the window roughly quadruples that part of the work — one reason windows have a fixed limit. And anything outside the window simply isn't there for the model on this pass: it can't see what scrolled off unless that text is fed back in. Long-context models soften the n² cost by having most layers attend only to a nearby window of recent tokens (sliding-window / local attention), keeping full all-to-all attention for just a few layers.