principles.fyi · the brain · concept

backpropagation

How the network figures out which way to nudge each knob to be less wrong.

blame for a weight = (how loss changes) × (how that weight changed things), chained backward; then nudge = −(small step) × blame

After the network makes a guess and we measure how wrong it was, backpropagation works out how much each tiny knob (weight) is to blame. It starts at the answer and traces the blame backwards, step by step, all the way to the start. Because it reuses the math it already did on the forward guess, it can figure out every knob's effect in one clean sweep instead of testing them one at a time. The blame tells you which direction makes things WORSE, so you nudge each knob a little the opposite way (and only a little) so next time the guess is a bit less wrong.