Synthetic Data Distillation

topic on 1 show · 2 statements across 1 episodes

2 statements about Synthetic Data Distillation, every show

Agarwal: Synthetic data distillation achieves 80% to 90% of target gains

“Try the simplest thing first, which is synthetic data distillation. That already gets you to 80% of the job or 90% of the job.”

Rishabh Agarwal Mar 23, 2025 ▶ 41:42 The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

LATENT SPACE Assertion Supported

Agarwal: Synthetic Data Distillation Can Outperform Logits on Benchmarks

“It's not always the case that synthetic data distillation beats, oh, sorry, it's worse than logits. Like, sometimes logits can be worse off. So if you look at the T-five base, two-fifty million scenario on GSM-HK, On the last plot, you can see that synthetic d…”

Rishabh Agarwal Mar 23, 2025 ▶ 26:20 The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

← every entity, every show

Turn any episode into a week of clips.

This entire site, thousands of episodes across every show transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Synthetic Data Distillation

2 statements about Synthetic Data Distillation, every show

Turn any episode into a week of clips.