The Overfitting Geometry Trap

Senior Machine Learning Engineer interview at Google DeepMind, and the interviewer asks:

“Your deep neural network achieves near-zero training loss but outputs absolute garbage in production. You plot it and see the network has learned a jagged, highly complex function perfectly threading a needle through your sparse training points. How does Early Stopping physically prevent the network from molding into this specific overfitting geometry?”

Don’t say: “Early stopping prevents overfitting because it monitors a hold-out validation set. Once the validation loss starts increasing, we halt training so the model doesn’t keep memorizing the noise in the training data.”