LLM System Design Interview, issue 13, Nov 10, 2025
The Weight Decay Illusion
ML Engineer interview at Anthropic, and the interviewer asks:
“You’re pre-training a 500B parameter model. You’re only doing one epoch over a 10T token dataset, so you’re clearly not overfitting. Why on earth are you still using weight decay?”
Don’t say: “Weight decay is a standard regularizer. It’s just good practice to prevent overfitting.”
When overfitting isn’t the problem, weight decay isn’t a regularizer - it’s an optimizer control knob shaping your convergence path.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.