synthetic data distillation

2 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Mar 23, 2025 by Rishabh Agarwal · across every show →

Everything said about synthetic data distillation, oldest first

Mar 23, 2025 neutral

Assertion Supported

Agarwal: Synthetic Data Distillation Can Outperform Logits on Benchmarks

“It's not always the case that synthetic data distillation beats, oh, sorry, it's worse than logits. Like, sometimes logits can be worse off. So if you look at the T-five base, two-fifty million scenario on GSM-HK, On the last plot, you can see that synthetic d…”

Rishabh Agarwal Mar 23, 2025 ▶ 26:20 The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

Mar 23, 2025 positive

Insight

Agarwal: Synthetic data distillation achieves 80% to 90% of target gains

“Try the simplest thing first, which is synthetic data distillation. That already gets you to 80% of the job or 90% of the job.”

Rishabh Agarwal Mar 23, 2025 ▶ 41:42 The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

synthetic data distillation

Everything said about synthetic data distillation, oldest first

Turn any episode into a week of clips.