continuous pre-training
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Aug 17, 2024 by Jeremy Howard · across every show →
Everything said about continuous pre-training, oldest first
Aug 17, 2024 neutral
Howard: Pre-training data mixes should be continuous per-batch functions, not discrete phases
“So the point at which they're doing proper continued pre-training is the point at which that becomes a continuum rather than a phase. So the only difference with what I was describing last time is to say, like, oh, they should, you know, There's a function or …”