Cosmopedia
2 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Dec 24, 2024 by Loubna Ben Allal · said 4 times in 1 episodes since 2024 · across every show →
Mentions by year
brought up most by Loubna Ben Allal (4)
tap a year for its mentions
Everything said about Cosmopedia, oldest first
Dec 24, 2024
Ben Allal: Prompt diversity is essential for scaling synthetic training data
“The key ingredient to getting a good data set that is synthetic is trying as much as possible to keep it diverse, because if you just throw the same prompts as your model, like generate, like, a textbook about linear algebra, and even if you change the tempera…”
Dec 24, 2024 positive
Ben Allal: LLMs can be trained with entirely synthetic pipelines
“Today you can train an LLM with like an entirely synthetic pipeline. For example, you can use our Cosmopedia data sets and you can train a one B model on like a hundred and fifty billion tokens. Those are a hundred percent synthetic, and those are also of good…”