Advanced Deep Learning Interview Questions, issue 11, Apr 1, 2026
The Bias-Weight Divergence Trap
Senior ML Engineer interview at Google DeepMind, and the interviewer asks:
“During debugging, you notice your biases are updating rapidly, but your weight matrices are completely frozen, despite both sharing the exact same upstream gradient vector from the next layer. Looking at the isolated backprop equations for weight gradients versus bias gradients, what specific forward-pass state is mathematically guaranteeing this failure?”
When biases update but weights freeze, it exposes a forward-pass collapse that most engineers mistakenly attribute to gradient instability.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.