The Causal Mask Leak

Senior ML Engineer interview at Google, and the interviewer asks:

“Your autoregressive model hits a suspiciously low training loss, but every generation is garbage. What’s the first thing you check in attention? And how can a −inf mask vs. a ‘large negative’ mask still give you NaNs or leaks under mixed precision?”

Don’t say: “Make sure the causal mask is applied with torch.triu and filled with −inf.”