Advanced Deep Learning Interview Questions, issue 19, Apr 9, 2026
The 1x1 Convolution Trap
Senior Computer Vision Engineer interview at Meta, and the interviewer asks:
“Your production CNN is hitting severe memory limits on your 80GB A100s. A junior engineer suggests replacing several 3x3 convolutions with 1x1 convolutions to “save space.”
Don’t say: “1x1 convolutions are a great optimization! They reduce the parameter count from 9 per channel down to 1. It acts as a dimensionality reduction layer, saving precious VRAM and compute while still extracting features.”
Replacing 3x3s with 1x1s silently removes the network’s ability to model local geometry, turning convolution into per-pixel channel mixing.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.