The SNR Collapse Trap

Senior AI Engineer interview at Midjourney, and the interviewer asks:

“Images are discrete RGB values from 0 to 255. Diffusion math assumes we sample from a standard normal distribution. If a data engineer feeds raw 0-255 pixel tensors directly into the training pipeline without continuous float scaling, how does this mathematically break the variance-preserving nature of the forward process?”

Don’t say: “It will cause exploding gradients. The unscaled inputs will saturate the activation functions, and your loss will immediately spike to NaN.”