step distillation
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Jun 1, 2026 by Ethan He · across every show →
Everything said about step distillation, oldest first
Jun 1, 2026 positive
He: Distillation works because teacher models are simpler than the internet
“I guess the, from the modeling perspective, the strong model, the teacher model is trying to model The image and videos of entire internet. And that distribution is extremely complex. As a step distilled model is just trying to learn from the teacher. The teac…”