Advanced Reinforcement Learning Interview Questions, issue 25, Feb 20, 2026
The Sparse Reward Trap
Senior AI Engineer interview at Anthropic, and the interviewer asks:
“You implemented Rejection Fine-Tuning (RFT) by sampling N solutions per problem and training on the correct ones. To push pass@1 accuracy, you drastically scale N, generating 100x more samples per prompt. Why does your test set error suddenly spike?”
When trajectory-level rewards ignore intermediate reasoning quality, scaling sample count turns lucky rollouts into systematic generalization failures.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.