principles.fyi · the brain · concept

derivative

The slope under your feet: which way is down, and how steep, measured purely locally.

f'(w) ≈ ( f(w + ε) - f(w) ) / ε for tiny ε

Zoom in on any smooth curve far enough and it stops curving — locally it is a straight line, and the derivative is that line's slope: how much the output nudges per tiny nudge of the input. It is a purely local measurement, needing no map of the whole landscape, yet it is enough to act on: step against the slope and the value decreases. Bundle every weight's personal slope into one vector and you have the gradient; the chain rule (local slopes multiply through composed functions) is what backpropagation exploits to compute them all in one backward sweep.