The QLoRA Compute Tax Trap

Senior AI Engineer interview at NVIDIA, and the interviewer asks:

“We switched from standard FP16 fine-tuning to QLoRA (4-bit quantization) to save memory. The model fits now, but training speed hasn’t improved, it’s actually slightly slower. Why didn’t reducing precision by 4x result in a 4x speedup?”

Don’t say: “That sounds like a bug. Since 4-bit integers are 4x smaller than 16-bit floats, we are moving less data. Less memory bandwidth usage always equals faster training.”