Advanced Reinforcement Learning Interview Questions, issue 16, Feb 11, 2026
The Bootstrapping Bias Trap
Senior RL Engineer interview at OpenAI, and the interviewer asks:
“We accidentally initialized our Value Network to output -1000 for every state. We run one update step using Monte Carlo and one using Bootstrapping (TD-Learning). Which algorithm breaks immediately, and which one survives?”
Bootstrapping doesn’t just reduce variance, it injects your model’s current errors directly into the label, turning bad initialization into self-reinforcing policy collapse.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.