Advanced Deep Learning Interview Questions, issue 14, Apr 4, 2026
The Dropout Scaling Trap
Senior ML Engineer interview at Meta, and the interviewer asks:
“You trained a large network with a heavy Dropout rate of 0.5. It performs flawlessly on the validation set. But when you export the raw weights to a custom offline C++ inference engine, the activations completely blow up and saturate. Assuming zero code bugs, what mathematical correction was missed?”
Don’t say: “You just forgot to disable dropout during inference using model.eval().”
Candidates forget dropout changes expected activation magnitude, so exporting raw weights without correcting the distribution silently breaks inference.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.