RAG Interview Questions, issue 23, Jul 27, 2026
The GraphRAG Evaluation Trap
Senior ML Engineer interview at Anthropic, and the interviewer asks:
“Your GraphRAG system beat vanilla RAG on summarization benchmarks. So why should I trust it on multi-hop reasoning?”
Don’t say: “Because graphs capture relationships, so multi-hop is where they shine.”
Why assuming community summaries equal logical traversal is a fatal benchmarking mistake, and how to avoid measuring recall when you should be testing precision under composition.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.