LLM System Design Interview, issue 41, May 4, 2026
The Latent Attention Trap
Senior LLM Engineer interview at DeepSeek, and the interviewer asks:
“You’ve implemented Multi-Head Latent Attention (MLA) to crush your KV-cache footprint. But uncompressing that latent vector requires an extra up-projection matrix, blowing up your inference FLOPs. How do you completely erase that computational cost entirely during the forward pass?”
Don’t say: “We can heavily quantize the up-projection matrix to INT8 or FP8 to speed up the operation, or use a custom fused Triton kernel to hide the latency behind memory bounds.”
Why uncompressing your KV-cache at runtime silently destroys your inference budget, and the linear algebra secret that lets you absorb the cost before weights even load into VRAM.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.