Calculus · Follow the steepest slope
Gradient descent on a loss
Place a starting point on an elongated bowl and step downhill. See why learning rate is a length, not a vibe.
Loss is an elongated bowl, f(x, y) = 0.25 x² + y². Click the field to place a start. Each step subtracts η ∇f. The bowl is stretched, so a naive step zigzags in y while crawling in x.
- x
- 2.200
- y
- 1.600
- f(x,y)
- 3.770
- ‖∇f‖
- 3.384
Small η crawls. Feature scaling is equivalent to making this bowl rounder.