Logit Based Distillation
topic on 1 show · 1 statements across 1 episodes
1 statements about Logit Based Distillation, every show
Agarwal: Logit Distillation Can Match Giant Teacher Models on Reasoning
“My hunch is that the logic-based distillation can go even further, and you might be able to even close the gap with the biggest of the teachers you have, because I don't think you need a huge number of parameters, because the reasoning process is very, very, l…”