Advanced Reinforcement Learning Interview Questions, issue 13, Feb 8, 2026
The Dead Gradient Trap
Senior Research Scientist interview at Google DeepMind, and the interviewer asks:
“We’re training an end-to-end Meta-RL agent to find objects in a procedurally generated house. The loss curves are completely flat, the agent isn’t learning to explore or solve the task. Why is the gradient dead, and what is the fundamental coupling failure happening here?”
When exploration and exploitation share the same objective, sparse rewards induce a chicken-and-egg failure that mathematically zeros out learning.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.