RAG Interview Questions, issue 15, Jul 19, 2026
The Ambiguity Relocation Trap
Senior ML Engineer interview at Anthropic, and the interviewer asks:
“You added an LLM query rewriter to fix ambiguous queries. Recall went up in your offline eval. So why did production accuracy quietly drop three weeks later?”
Don’t say: “The rewriter must have a bug”
How collapsing queries into a single confident guess tricks your evals while destroying intent fidelity, and why the fix requires rewrite candor and late-stage arbitration.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.