The Quantization Gradient Trap

Senior AI Engineer interview at Meta, and the interviewer asks:

“We need to switch to 𝐐𝐮𝐚𝐧𝐭𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐀𝐰𝐚𝐫𝐞 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠 (𝐐𝐀𝐓) because post-training quantization is tanking our accuracy. But the rounding operation (Float -> Int8) is a step function with a derivative of zero. How do you actually backpropagate gradients through it to update the weights?”