Rishabh Agarwal
2 statements across 1 episodes · 2 bullish · 0 bearish · 1 people on the record · first statement Mar 23, 2025 by Rishabh Agarwal · across every show →
On the record as a speaker too: Rishabh Agarwal's record, appearances and statements → this page counts the times other people say the name.
Everything said about Rishabh Agarwal, oldest first
Mar 23, 2025 positive
Agarwal: Distilling a Large Model Outperforms Direct Training on the Same Data
“This is something that people have found again and again, that basically you can train a model on some data, or you can train a bigger model on that data and distill that model to another model, and that distill model is better.”
Mar 23, 2025 positive
Agarwal: 50/50 mix of KL divergences usually works when goals are unclear
“The general recommendation I would give people is that maybe use a mixture of half and half. Like that's what some people have used, right? That's like saying, yeah, basically saying, I don't know what I want. I just want something to work well enough. I'll ju…”