data mix scheduling
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Jun 25, 2024 by Jonathan Frankle · across every show →
Everything said about data mix scheduling, oldest first
Jun 25, 2024 positive
Frankle: Dynamic data mixing during pre-training is effective for domain-specific models
“We've had some surprisingly good luck with this. We just released a paper on it. The details matter a lot and it really matters what you're trying to do with the model. But it's been quite effective for us depending on the setting. And certainly when we're thi…”