principles.fyi · the brain · concept

parameters

The model's learned numbers — the knobs it tunes during training.

output = sum_i (weight_i * input_i) + bias

Parameters are the numbers (mostly weights, plus biases) inside the model that get adjusted during training; "405B" means 405 billion of them. Each one is a tiny dial that nudges how an input signal flows toward an output, so the model is really one huge stack of multiply-and-add operations governed by these numbers. In a transformer they live in the attention layers (deciding which words look at which), the feed-forward layers (which act like memory, storing patterns and facts), and the embedding tables (turning tokens into vectors and back). Training slowly tunes every dial to lower prediction error. More parameters means more room to store patterns — which is part of why bigger models can do more.