LLM System Design Interview, issue 10, Nov 8, 2025
The Thinking Mode Fusion Trick
Senior ML Engineer interview at Google DeepMind, and the interviewer asks:
“Our new reasoning model is great, but it uses a 2000 token Chain of Thought even for simple questions like ‘What is 2+2?’. This is killing our inference budget. How do you fix this without sacrificing its ability to solve complex problems?”
Don’t say: “I’d train two models, a small, fast one for simple queries and our large reasoning model for hard ones.”
Why running two models is a maintenance nightmare - and how to teach one model when to think and when to act, saving your inference budget without losing reasoning power.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.