Advanced Reinforcement Learning Interview Questions, issue 15, Feb 10, 2026
The Cold Start Exploration Trap
Senior AI Research Engineer interview at Google DeepMind, and the interviewer asks:
“We are training a new RL agent to manipulate a robot arm for a task like pouring water. A junior engineer suggests initializing with standard epsilon-greedy exploration to discover the first high-reward state. Why is this mathematically doomed, and what is the production-ready alternative?”
Exploration-from-scratch collapses in continuous action spaces, forcing production systems to cheat by initializing inside the reward manifold via demonstrations.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.