The View vs Copy Trap

Senior ML Engineer interview at Google DeepMind, and the interviewer asks:

“You are passing a transposed matrix into a custom loss function to fix a shape mismatch. Your code throws a contiguity error, so you add a .reshape() or .contiguous().view() to make it compile. Suddenly, your PyTorch profiler shows a massive, unexpected spike in VRAM usage. What silent memory allocation trap did you just trigger?”

Don’t say: “It’s probably an issue with Python’s garbage collector holding onto the old tensor graph, or maybe the custom loss function is accidentally tracking gradients for the labels. I’d just scatter some torch.cuda.empty_cache() calls before the loss.”