Advanced Deep Learning Interview Questions, issue 15, Apr 5, 2026
The Convexity Assumption Trap
Senior ML Engineer interview at Meta, and the interviewer asks:
“You’re migrating a legacy continuous prediction model into a multi-class classifier. A junior dev suggests keeping the L2 (MSE) loss for the new Softmax outputs because ‘error is error.’ Why is this guaranteed to break the optimizer in production?”
Don’t say: “Because Cross-Entropy is meant for probabilities and MSE is for regression.”
Many assume the optimizer sees a smooth bowl, but MSE over Softmax outputs produces a jagged, non-convex surface that standard training loops can’t reliably descend.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.