StarZero All shows Try the podcast clipper →

FineWebEDU

0 statements across 0 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Dec 24, 2024 by Loubna Ben Allal · across every show →

Everything said about FineWebEDU, oldest first

Dec 24, 2024 neutral
Disclosure
Hugging Face Filtered 15T Token Dataset Down to 1.5T Educational Tokens
“And then we run this classifier on all of fine web, which is a 15 trillion tokens data set. And then we only keep the pages that have like a score that's higher than three. So for example, in our case, we went from 15 trillion tokens to just 1.5 trillion token…”
Loubna Ben Allal Dec 24, 2024 ▶ 12:21 Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
Dec 24, 2024 bullish
Assertion Supported
Ben Allal: FineWeb-Edu Outperforms All Other Public Web Datasets
“And as you can see here FineWebEDU outperforms all the other public web datasets by a larger margin on a couple of benchmarks.”
Loubna Ben Allal Dec 24, 2024 ▶ 12:37 Best of 2024: Synthetic Data / Smol Models, Loubna Ben Allal, HuggingFace [LS Live! @ NeurIPS 2024]
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more