The Backprop Routing Trap

Senior AI Engineer interview at Meta, and the interviewer asks:

“You wrote a custom, bare-metal CUDA Max Pooling operation that cuts inference latency by 40%. But when you drop it into the training loop, gradient descent completely breaks. Why?”

Don’t say: “It’s a numerical stability issue. You probably aggressively quantized to FP16 or INT8 for the inference optimization, which caused underflow or vanishing gradients during the backward pass. You just need to upcast back to FP32 or BF16 for training.”