The Gradient Shockwave Trap

Senior ML Engineer interview at NVIDIA, and the interviewer asks:

“You attach a new, random linear head to a pre-trained Transformer. Do you unfreeze all layers and start backprop immediately?”