Curriculum Learning
topic on 2 shows · 3 statements across 3 episodes
3 statements about Curriculum Learning, every show
Curriculum learning enables multi-physics AI fine-tuning with significantly fewer samples
“And now there's coupling, like because of heat, there's also stretching or kind of the joint phenomena. You could like now hope to fine tune with much fewer samples because it kind of individually knows this phenomena. Then combining them together, maybe it ca…”
Aidan Gomez: Curriculum learning has failed in machine learning
“What's funny is that curriculum learning has actually failed in machine learning. We don't really do curriculum learning. It's just throw the hardest material and the easiest material all at the same time and let the model figure it out.”
Huang: Curriculum context expansion outperforms full-length training from scratch
“If you train a model on a shorter context and you progressively increase that context to, like,
You know, the final limit that you have, like, 32 K is usually the limit of Lama two was that long.
It actually performs better than if you try to train 32 K the …”