small language models
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Dec 24, 2024 by Loubna Ben Allal · across every show →
Everything said about small language models, oldest first
Dec 24, 2024 positive
Ben Allal: Distilling from large models avoids small-model self-training collapse
“I think if you do that approach, it's normal to observe this kind of behavior because the quality is going to be worse because the model is already small, and then if you train it just on these generations, you shouldn't expect it to become better. But what we…”