LLM System Design Interview, issue 53, Aug 28, 2026
The Fallback Token Paradox
Senior ML Engineer interview at OpenAI, and the interviewer asks:
“Your tokenizer fails to round-trip on 0.3% of production traffic, emoji, mixed-script usernames, truncated UTF-8 from a bad client. Your teammate maps them all to UNK and ships. What did he just break?”
Don’t say: “Nothing much, it’s only 0.3% of traffic, the model will learn to ignore it.”
The hidden failure where covering your edge cases physically prevents your model from learning them, and the byte-level trick GPT-2 and Llama use to fix it.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.