LLM Inference Interview Questions, issue 21, Aug 21, 2026
The Search Error Trap
Senior ML Engineer interview at Google DeepMind, and the interviewer asks:
“Your colleague says greedy decoding is a fine approximation of the most likely sequence, because we take the max at every step. Where does that argument break, and does beam search actually fix it?”
Don’t say: “Greedy is fine, beam search finds the highest-probability sequence.”
Why your decoder is quietly throwing away the model's correct answers, and the counterintuitive reality of why reducing search error actively degrades user experience.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.