LLM Agents Interview Questions, issue 11, Mar 5, 2026
The Lost-in-the-Middle Trap
Senior AI Engineer interview at Stripe, and the interviewer asks:
“You’re building a complex text-to-SQL agent. You stuff all 50 database grammar rules into an 8k system prompt for Chain-of-Thought reasoning, but the model constantly loses track of constraints and hallucinates table joins. Why?”
Don’t say: “The context window isn’t big enough. We need to upgrade to a 128k or 1M token context window, or we just need to add more few-shot examples to the prompt.”
Even with massive context windows, transformer attention degrades when critical constraints sit among thousands of irrelevant tokens.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.