Advanced NLP Interview Questions, issue 6, Dec 13, 2025
The LoRA Initialization Trap
Senior ML Engineer interview at Google DeepMind, and the interviewer asks:
“You are implementing 𝐋𝐨𝐑𝐀 (𝐋𝐨𝐰-𝐑𝐚𝐧𝐤 𝐀𝐝𝐚𝐩𝐭𝐚𝐭𝐢𝐨𝐧) from scratch. How do you initialize the down-projection matrix A and the up-projection matrix B?”
How random initialization injects noise into frozen weights, and why zero is the safest starting point.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.