Why Your 1B → 70B Training Exploded

AI Engineer interview at Google DeepMind, and the interviewer asks:

“Your 1B parameter proxy model trains perfectly with a 1.2e-4 learning rate. You scale the model to 70B, and the training immediately explodes. What’s the most 𝘭𝘪𝘬𝘦𝘭𝘺 reason and how do you fix it 𝐰𝐢𝐭𝐡𝐨𝐮𝐭 running a new, expensive hyperparameter sweep?”

Don’t say: “The model is too big, so the updates are unstable. I’d add gradient clipping and just keep lowering the learning rate manually until it’s stable.”