LLM System Design Interview, issue 5, Nov 5, 2025
The FLOPs Compression Hack
AI Engineer interview at OpenAI, and the interviewer asks:
“Why don’t we just use a simple byte-based tokenizer? It has a fixed 256-token vocabulary, it’s simple, and it never has an ‘unknown’ token. Why are we still using a complex BPE tokenizer?”
Don’t say: “Because BPE groups related characters into subwords that have more semantic meaning...”
Byte tokenizers look simple - until they 16x your attention cost. The real reason every frontier model still relies on subword compression.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.