LLM System Design Interview, issue 55, Aug 30, 2026
The LayerNorm Trap
Senior LLM Training Engineer interview at Anthropic, and the interviewer asks:
“Your profiler says LayerNorm is 0.17% of your FLOPs but 25% of your step time. Do you optimize it?”
Don’t say: “No, 0.17% of compute isn’t worth the engineering time. I’d go optimize the attention kernels instead.”
When optimizing for compute leaves your GPU starving for memory, and the kernel fusion trick that dictates whether RMSNorm is a massive speedup or a rounding error.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.