Rishabh Agarwal
person on 1 show · 2 statements across 1 episodes
On the record as a speaker too: Rishabh Agarwal across every show, appearances and statements → this page counts the times other people say the name.
2 statements about Rishabh Agarwal, every show
Agarwal: 50/50 mix of KL divergences usually works when goals are unclear
“The general recommendation I would give people is that maybe use a mixture of half and half. Like that's what some people have used, right? That's like saying, yeah, basically saying, I don't know what I want. I just want something to work well enough. I'll ju…”
Agarwal: Distilling a Large Model Outperforms Direct Training on the Same Data
“This is something that people have found again and again, that basically you can train a model on some data, or you can train a bigger model on that data and distill that model to another model, and that distill model is better.”