Machine Learning System Design Interview, issue 29, May 17, 2026
The Correlational Trap
Senior ML Engineer interview at Meta, and the interviewer asks:
“Your consumer app’s new recommendation model increased short-term engagement metrics (like “time spent”
Don’t say: “We just need to adjust our loss function. I’ll add a penalty term for clickbait, or pull historical log data to find a new proxy metric that correlates better with 30-day retention. We can just retrain the model on that new proxy.”
Why relying on offline logs creates a hidden death spiral for user retention, and how counterfactual analysis separates true value from algorithmic addiction.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.