The Benchmark Autonomy Trap

Senior AI Engineer interview at Google, and the interviewer asks:

“A year ago you shipped a fixed LLM workflow: hard-coded vendor list, fixed steps, pick the cheapest. A newer model now runs the whole task as a free-form agent and beats it on your benchmark. Do you delete the workflow?”

Don’t say: “Yes. The model got smarter, so the scaffolding is dead weight.”