LLM Inference Interview Questions, issue 12, Aug 10, 2026
The Top-k Distractor Trap
Senior AI Engineer interview at Anthropic, and the interviewer asks:
“Your agent has 3,000 registered APIs, so you retrieve top-k tool specs into context per query. It starts confidently calling the wrong tool. Where’s the failure, and why is bumping k the wrong first move?”
Don’t say: “The model is confused, so I’d retrieve more candidates and improve the prompt.”
Why feeding your agent more API choices quietly destroys selection precision, and how adding a simple abstention path stops silent substitution in its tracks.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.