Advanced Reinforcement Learning Interview Questions, issue 20, Feb 15, 2026
The Static CoT Trap
Principal AI Engineer interview, and the interviewer asks:
“We’re building a reasoning model like DeepSeek R1. We want the model to burn test-time compute exploring solutions for complex math, but answer instantly for ‘2+2’. How do you formulate the RL objective to achieve this adaptive behavior?”
Training on reasoning traces without a compute-aware reward turns the model into a pattern imitator, not an agent that allocates inference dynamically.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.