LLM System Design Interview, issue 64, Sep 8, 2026
The LR Extrapolation Trap
Staff ML Engineer interview at Anthropic, and the interviewer asks:
“You ran one big grid search over learning rate and batch size. The batch-size scaling law is a gorgeous straight line in log-log. The learning-rate fit is a shotgun blast with a trendline drawn through it out of politeness. Same runs, same data. Why?”
Don’t say: “We need more grid points”
Why fitting power laws over a 0.5-decade dynamic range is just squinting at noise, and how to replace fragile LR sweeps with guaranteed parameter transfer.
The full answer, with the mechanism and the arithmetic, is free on Substack.