Issue 10, Sep 29, 2026
The Prompt Boundary Tokenizer Trap
Senior AI Engineer interview at Microsoft, and the interviewer asks:
“A teammate retrained our code tokenizer to merge freely across whitespace. Sequences got ~40% shorter and held-out loss improved. Why might IDE autocomplete get worse after we ship it, and what do you check before approving?”
Don’t say: “Shorter sequences plus lower loss means a better model. Ship it.”
Why lower full-file loss silently wrecks half-typed completions, and how token healing and bits-per-byte metrics catch the flaw before deployment.
Share this trapShare on LinkedIn
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.