Advanced NLP Interview Questions, issue 10, Dec 16, 2025
The Contrastive Batch Size Trap
Machine Learning Engineer interview at OpenAI, and the interviewer asks:
“We need to train a specialized CLIP model for medical imaging from scratch. You have a node of 8 A100s. What batch size do you configure?”
In CLIP, small batches don’t just slow training, they mathematically break the loss.
The full answer, with the mechanism and the arithmetic, is for paid subscribers on Substack.