principles.fyi · the brain · concept

expected value

Each outcome weighted by how often it happens — the number the long-run average is forced toward.

E[X] = Σ p(x) · x

The expectation E[X] sums every possible outcome times its probability. It is not a prediction of the next draw (which stays random) but a property of the whole distribution: by the law of large numbers, the running average of repeated draws converges to it, with per-draw error shrinking like 1/sqrt(n). Flipped around, this is the license for sampling — you can estimate an average over an untouchably large distribution by averaging a few random draws — which is why models can train on batches, why RLHF can maximize expected reward from sampled generations, and why noisy gradient estimates still walk downhill on average.