Issue 3, Oct 17, 2026
The Softmax Confidence Trick
AI Engineer interview at Microsoft, and the interviewer asks:
“You auto-approve anything your classifier scores above 0.9 after the softmax. Product adds a fourth label, accuracy holds, and the auto-approve queue fills with junk. What was that 0.9 actually measuring?”
Don’t say: “0.9 is the model’s confidence. The threshold just needs retuning for four classes.”
A 0.9 after the softmax is a vote share, not a verdict and adding a fourth label silently moves the gate you tuned on three while accuracy holds steady.
Share this trapShare on LinkedIn
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.