Gradient descent on a loss

Loss is an elongated bowl, f(x, y) = 0.25 x² + y², so ∇f = (0.5 x, 2 y). Click the field to place a start. The arrow is the next step, −η ∇f; the small ring is where the dot will land. Curvature is 4× larger along y, so one η cannot suit both axes.

x

2.200

y

1.600

f(x,y)

3.770

‖∇f‖

3.384

steps

0

η ≤ 0.5: both shrink every step — y by ×0.30, x by only ×0.82. The long axis is the slow one. Feature scaling makes this bowl rounder.