training data mix
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Jun 25, 2024 by Josh Albrecht · across every show →
Everything said about training data mix, oldest first
Jun 25, 2024 negative
Albrecht: Dynamic data mix schedules during LLM training yield negligible gains
“We did some experiments and we've actually talked to a bunch of researchers who were doing work here as well and looking at kind of their experiments on this. And we were originally pretty hopeful because it sounds like something that should work and make sens…”