distillation
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Jan 31, 2025 by Bob McGrew · across every show →
Everything said about distillation, oldest first
Jan 31, 2025 positive
McGrew: Frontier labs will increasingly rely on model distillation for smaller models
“I think over the last year, the big frontier labs and a lot of other people have figured out the tricks to take big models. And, you know, take a very particular distribution of user input and train a model that is almost as good as the big model, but much, mu…”