KV cache, batching, decoding, and the agents that run on top of them.
25 traps, Jul 2026 to Aug 2026. Complete.
Set in interviews at Anthropic (16), Google DeepMind (4), Google (2) and Meta (1).
Each trap: the interviewer’s question, the answer most candidates give, and the mechanism that breaks it. The full answers are on Substack.
Senior AI Engineer interview at Stripe
“Your LATS agent crushes HotpotQA. Now point it at a checkout flow where one of the actions is ‘submit payment.’ How do you keep tree search from becoming a production incident?”
LLM Inference #25
Aug 25, 2026
Senior AI Engineer interview at Anthropic
“Your agent completes 59-minute tasks at 50% success. Leadership wants to ship it autonomously. What’s your deployment boundary?”
LLM Inference #24
Aug 24, 2026
Senior AI Engineer interview at Google DeepMind
“Your agent retrieves mid-reasoning instead of once upfront. Recall@10 went up. Answer accuracy went down. Where’s the failure?”
LLM Inference #23
Aug 23, 2026
Senior AI Engineer interview at Anthropic
“Your agent triggers a web search whenever its reasoning trace shows uncertainty, ‘perhaps,’ ‘wait,’ ‘alternatively.’ It works great on your eval set and misses the worst failures in production. Why?”
LLM Inference #22
Aug 22, 2026
Senior ML Engineer interview at Google DeepMind
“Your colleague says greedy decoding is a fine approximation of the most likely sequence, because we take the max at every step. Where does that argument break, and does beam search actually fix it?”
LLM Inference #21
Aug 21, 2026
Staff ML Engineer interview at Google DeepMind
“AlphaCode picked submissions by clustering programs on behavioral equivalence. AlphaCode 2 used a fine-tuned scoring model instead. Your junior wants to rip out the clustering and ship the reward model tomorrow. What breaks?”
LLM Inference #20
Aug 20, 2026
Senior AI Engineer interview at Google DeepMind
“You 10x’d your sampling budget on a hard reasoning task and the solve rate barely moved, even though the paper’s log-linear scaling curve says it should have. What’s the first thing you measure?”
LLM Inference #19
Aug 19, 2026
ML Engineer interview at Anthropic
“Best-of-16 rollouts with a critic reranker moves your agent from 20% to 32% on SWE-bench. Your PM wants it shipped. What do you tell him?”
LLM Inference #18
Aug 18, 2026
Staff AI Engineer interview at Anthropic
“Your agent scores 12 points higher with reasoning effort set to high, and 40% of users abandon the session before it finishes. How do you decide where to spend that reasoning budget?”
LLM Inference #17
Aug 17, 2026
Senior ML Engineer interview at Anthropic
“You’re training a tool-use agent. Do you SFT on expert tool-call traces before RL, or go straight to RL from the base model?”
LLM Inference #16
Aug 15, 2026
Staff AI Engineer interview at OpenAI
“You’re running RL to teach a model tool use. Your correctness reward gives partial credit, overlap on tool names, then parameter names, then parameter values. What does the policy learn to exploit before it learns to call tools correctly?”
LLM Inference #15
Aug 14, 2026
Senior AI Engineer interview at Anthropic
“Your agent calls a broken file-editing tool 11 times in a row with the same malformed params. You add ‘if a tool fails, note it and try another approach.’ Failure detection improves. Task success doesn’t. Why?”
LLM Inference #14
Aug 13, 2026
Staff AI Engineer interview at Google
“You shipped a code-executing agent behind an AST sandbox that blocks os, subprocess, and dunder access. Security signed off. Six weeks later you’re breached. What class of attack did your sandbox structurally fail to stop?”
LLM Inference #13
Aug 11, 2026
Senior AI Engineer interview at Anthropic
“Your agent has 3,000 registered APIs, so you retrieve top-k tool specs into context per query. It starts confidently calling the wrong tool. Where’s the failure, and why is bumping k the wrong first move?”
LLM Inference #12
Aug 10, 2026
Senior ML Engineer interview at Meta
“You’re building synthetic tool-call training data with Toolformer’s filter, keep the call if the tool output raises the likelihood of the correct continuation. Your eval improves. Production accuracy doesn’t move. What’s wrong with the filter?”
LLM Inference #11
Aug 9, 2026
Senior AI Engineer interview at Anthropic
“Your coding agent keeps failing on multi-line Python, escaped quotes, mangled indentation. A teammate says just use a stronger model. What’s the real fix?”
LLM Inference #10
Aug 8, 2026
Senior AI Engineer interview at Anthropic
“Your agent has 40+ tools registered across five MCP servers. Accuracy is worse than when it had 8. Your team says ‘the model got confused.’ What’s actually degrading, and what do you strip versus route?”
LLM Inference #9
Aug 7, 2026
Staff AI Engineer interview at Anthropic
“Your team wants to split your agent into a planner, a coder, and a QA agent, ‘because that’s how real software teams work. Would you ship it?”
LLM Inference #8
Aug 6, 2026
Senior AI Infrastructure Engineer interview at Anthropic
“You enabled prompt caching on a 100-step agent trajectory expecting a 5–10x cost drop. In production you’re seeing barely 1.3x, and most per-step content is cache-missing. What’s actually happening, and why does the order of your context decide whether caching works at all?”
LLM Inference #7
Aug 5, 2026
Senior AI Engineer interview at Google
“Your web agent uses set-of-marks, you screenshot the page, draw numbered boxes on every element, and let the VLM click by number. It works in your demo but in production the model keeps clicking box 41 when it meant box 14, and it ignores half the page. The VLM is state-of-the-art and multimodal. So why is it failing?”
LLM Inference #6
Aug 4, 2026
AI Agents Engineer interview at Anthropic
“You want to evaluate whether your coding agent actually solved a task. A teammate says: skip the learned critic, just train a model to generate unit tests and let pass/fail decide. It’s system-agnostic and elegant. Why isn’t this the default?”
LLM Inference #5
Aug 3, 2026
Senior AI Engineer interview at Anthropic
“You’re reranking 16 agent trajectories with a critic model to boost your SWE-bench score. Accuracy went up. Now walk me through the exact shape of that accuracy-vs-rollouts curve, and tell me when this 16x spend is actually worth it in production.”
LLM Inference #4
Aug 2, 2026
Senior AI Engineer interview at Anthropic
“Your coding agent uses a task-tracker tool, and the current plan is already in its output. So why do you also write that plan to a tasks.md file on disk, what does that actually buy you?”
LLM Inference #3
Aug 1, 2026
AI Agent Engineer interview at Anthropic
“Your coding agent keeps overflowing the context window. You add LLM summarization to condense old steps, cost drops 2x on SWE-bench. But now the agent opens the same pull request three times. What did summarization actually break, and how do you fix it without losing the savings?”
LLM Inference #2
Jul 31, 2026
Senior ML Engineer interview at Anthropic
“Your self-correction loop improves output quality on every pass, but p99 latency just tripled in prod. Your teammate says ‘just cap the max iterations.’ Why is that a band-aid, and what’s actually broken?”
LLM Inference #1
Jul 30, 2026