RAG Interview Questions, issue 14, Jul 18, 2026
The Iterative Retrieval Trap
Staff ML Engineer interview at Anthropic, and the interviewer asks:
“Your iterative RAG boosted multi-hop accuracy, but p99 latency tripled and inference costs are bleeding. A PM suggests capping retrieval at 3 loops. Why is that the wrong fix, and what should actually decide when to stop?”
Don’t say: “We’ll set max_iterations to 3 to bound the latency.”
Why a fixed retrieval cap quietly bleeds inference costs on zero-hop queries, and how to predict query complexity before you spend a single retrieval.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.