The muP Transfer Trap

Senior AI Research Engineer interview at Anthropic, and the interviewer asks:

“Your team burned three weeks sweeping learning rates at every rung of the scaling ladder before the 7B run. A colleague says muP would have let you tune once at 100M and transfer for free. Were they right?”

Don’t say: “Yes, muP makes the optimal learning rate width-invariant, so you tune small and transfer up.”