The Transformation Debt Trap

Senior ML Engineer interview at Meta, and the interviewer asks:

“We need to ingest petabytes of raw, unstructured data, text, images, and audio. for our new multimodal GenAI pipeline. Everyone loves the modern data stack, so should we use ELT to dump it all into the data lakehouse as fast as possible and transform it later?”

Don’t say: “Absolutely. ELT is the modern standard. Storage is cheap, so we should extract the raw data, load it immediately to avoid data loss, and use dbt or Spark to run transformations on the fly inside the warehouse. It gives us maximum flexibility for exploratory model training.”