Advanced NLP Interview Questions, issue 8, Dec 14, 2025
The Dropout Inference Trap
Senior Machine Learning Engineer interview at OpenAI, and the interviewer asks:
“We implemented a custom Dropout layer from scratch. How do you handle it during inference?”
Why “just turning it off” silently doubles your activations and destroys production predictions.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.