Sources

The worked examples in Partial derivatives and backpropagation can be checked directly. These sources explain the underlying methods and their implementation.

• Stanford CS231n: Backpropagation, Intuitions — scalar operations, local derivatives, and vector-valued calculations.
• Baydin et al. (2018), Automatic Differentiation in Machine Learning: a Survey — forward and reverse mode, computational cost, and implementation.
• PyTorch: Autograd mechanics — saved values, gradient accumulation, and behavior at nondifferentiable points.
• Goodfellow, Bengio, and Courville (2016), Deep Learning, chapter 6 — feed-forward networks and backpropagation.
• Rumelhart, Hinton, and Williams (1986), Learning representations by back-propagating errors — the original research article.
• Griewank and Walther (2008), Evaluating Derivatives — a detailed treatment of algorithmic differentiation.
• 3Blue1Brown: Backpropagation calculus — a visual account of the chain rule through a network.