Morcos: Data curation still has at least 100x in performance gains ahead
Ari Morcos · Better Data is All You Need — Ari Morcos, Datology · Aug 29, 2025 · at 32:23
Ari Morcos, CEO of Datology, predicts the remaining headroom in data curation efficiency for AI models.
0:00 / 0:05exact quote · 5.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You know, we've already been able to get 10 X gains. I think there's at least another hundred X behind this that are still to be done.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ari Morcos
Morcos: The Transformer is just one of many equivalently good architectures
“And one of my like more controversial viewpoints, I think, is that I think the transformer is a great advance to be sure, but I think it's one of a very large Set of equivalently good architectures that we could have found. And there are many, many ways we cou…”
Morcos: GPT-4.5 and Llama 4 show limits of naive mega-model scaling
“And I think that's what we've seen to some extent with the failure of the mega models, right? With 4.5 and Lama four and others. I think that there is a challenge of just continuing to do that naively and you have to figure out how to break it.”
Morcos: Post-training alignment is ineffective long-term compared to pre-training alignment
“Like fundamentally, I think alignment and post training doesn't really make sense as a long-term solution. If you can easily align a model through post training, you can easily misalign a model through post training. If it's easy to put it in, it's easy to tak…”
Morcos: Soft inductive biases become harmful past 1M data points in vision
“Turns out in the small data regime, and when I say small data here, I mean, say less than 500,000 data points. And this was in the context of image self-supervised learning. So in that small data regime, this is super helpful. And where this paper's actually b…”