StarZero All shows Try the podcast clipper →

model collapse

3 statements across 2 episodes · 3 bullish · 0 bearish · 2 people on the record · first statement Dec 24, 2024 by Loubna Ben Allal · across every show →

Everything said about model collapse, oldest first

Dec 24, 2024 positive
Prediction Not checkable as stated
Ben Allal: Properly curated synthetic data prevents model collapse
“And I think there's a lot of concerns about model collapse, and I'm going to talk about that later, but we'll see that like, if we use synthetic data properly and we curate it carefully that shouldn't happen.”
Loubna Ben Allal Dec 24, 2024 ▶ 2:09 Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
Dec 24, 2024 positive
Insight
Ben Allal: Distilling from large models avoids small-model self-training collapse
“I think if you do that approach, it's normal to observe this kind of behavior because the quality is going to be worse because the model is already small, and then if you train it just on these generations, you shouldn't expect it to become better. But what we…”
Loubna Ben Allal Dec 24, 2024 ▶ 4:56 Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
Aug 29, 2025 positive
Insight
Morcos: Filtering synthetic data between generation cycles prevents model collapse
“If you filter the data after each point, that's now information injection, and that can break all of this and I think can prevent model collapse.”
Ari Morcos Aug 29, 2025 ▶ 43:47 Better Data is All You Need — Ari Morcos, Datology
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more