The Max Pooling Gradient Trap

Senior Computer Vision Engineer interview at Meta, and the interviewer asks:

“You’re using a Max activation function across a set of feature maps. During backpropagation debugging, you notice that the vast majority of your weights in the preceding layer aren’t updating at all. Why is this mathematically expected, and how does the engine handle exact ties?”

Don’t say: “It sounds like a vanishing gradient problem or a dying ReLU issue. I would just bump up the learning rate, or switch to Average Pooling so the gradients can flow back to all the weights evenly.”