Training Data Mix
topic on 1 show · 1 statements across 1 episodes
1 statements about Training Data Mix, every show
Albrecht: Dynamic data mix schedules during LLM training yield negligible gains
“We did some experiments and we've actually talked to a bunch of researchers who were doing work here as well and looking at kind of their experiments on this. And we were originally pretty hopeful because it sounds like something that should work and make sens…”