The VRAM Bottleneck Trap

Senior AI Engineer interview at Meta, and the interviewer asks:

“Instead of relying on 𝘗𝘺𝘛𝘰𝘳𝘤𝘩'𝘴 𝘣𝘶𝘪𝘭𝘵-𝘪𝘯 𝘢𝘶𝘵𝘰𝘨𝘳𝘢𝘥 𝘦𝘯𝘨𝘪𝘯𝘦, in what highly constrained production scenario does writing 𝘤𝘶𝘴𝘵𝘰𝘮 𝘧𝘰𝘳𝘸𝘢𝘳𝘥 𝘢𝘯𝘥 𝘣𝘢𝘤𝘬𝘸𝘢𝘳𝘥 𝘱𝘢𝘴𝘴𝘦𝘴 𝘧𝘳𝘰𝘮 𝘴𝘤𝘳𝘢𝘵𝘤𝘩 become an absolute engineering necessity?”

Don’t say: “It's to understand the underlying math better, or maybe to implement a custom mathematical function that isn't naturally differentiable.”