Generative Vision Interview Questions, issue 18, Jun 26, 2026
The REPA Alignment Trap
Senior ML Engineer interview at OpenAI, and the interviewer asks:
“You added a REPA loss to your diffusion transformer to speed up training. It barely helped. What did you get wrong?”
Don’t say: “REPA just doesn’t work that well, the speedups in the paper are cherry-picked.”
Why adding representation alignment to your diffusion transformer quietly wastes an 18x training speedup, and how matching abstraction levels saves your compute budget.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.