RAG Interview Questions, issue 20, Jul 24, 2026
The Feedback Loop Trap
Staff ML Engineer interview at Google, and the interviewer asks:
“Your REALM-style retriever learns from its own retrieval signal, no human labels. What breaks after six months in production?”
Don’t say: “Nothing, it’s self-supervised, it keeps improving as it sees more data.”
Why self-taught retrievers quietly destroy their own retrieval distribution while your eval metrics stay green, and the elite engineering shift from tracking accuracy to monitoring retrieval entropy.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.