Step Distillation
topic on 1 show · 1 statements across 1 episodes
1 statements about Step Distillation, every show
He: Distillation works because teacher models are simpler than the internet
“I guess the, from the modeling perspective, the strong model, the teacher model is trying to model The image and videos of entire internet. And that distribution is extremely complex. As a step distilled model is just trying to learn from the teacher. The teac…”