The Near-Duplicate Leakage Trap

Senior ML Engineer interview at Google DeepMind, and the interviewer asks:

“You scraped GitHub, held out a random 5% of files as your test set, and your new code model just posted a big jump on it. Before you announce anything, what’s the first thing you’d suspect about how that split was built?”

Don’t say: “Overfitting. I’d check the training curves and add regularization.”