epoching
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Aug 29, 2025 by Ari Morcos · across every show →
Everything said about epoching, oldest first
Aug 29, 2025 positive
Morcos: Epoching high-quality data beats training on new average data
“Repeating higher quality tokens is almost always better than seeing net new lower quality tokens. So, like, epoching over higher quality data, almost always better than getting the same amount of new data of an unknown quality, or of average quality, average i…”