pre-training

3 statements across 3 episodes · 2 bullish · 1 bearish · 3 people on the record · first statement Apr 25, 2025 by Logan Kilpatrick · across every show →

Everything said about pre-training, oldest first

Apr 25, 2025 bullish

Opinion

Kilpatrick: Pre-Training Isn't Dead; Gains Multiply Through Post-Training and RL

“And this is why, like, I don't subscribe to the, like pre-training is, you know, dead and all that stuff, because the more work that you can do at the pre-training level, those capabilities, as you do post-training and as you give the models RL capability, it'…”

Logan Kilpatrick Apr 25, 2025 ▶ 12:50 Google's AI Comeback in Their Own Words - Logan Kilpatrick

Jul 8, 2025 bearish

Opinion

Patel: Pre-training returns are plateauing; GPT-4.5 was unimpressive and deprecated

“Pre-training seems to have been giving us these plateauing returns. We make these models bigger. GPT-Fort .5 didn't seem to be all that impressive. They had to deprecate it.”

Dwarkesh Patel Jul 8, 2025 ▶ 5:24 DWARKESH PATEL on the Biggest Problem of AI

Oct 28, 2025 positive

Insight

Nadella: Pre-training Remains More Efficient Than RL Due to Amortization

“Pre-training is a more efficient form of training. Because you can advertise it.”

Satya Nadella Oct 28, 2025 ▶ 11:33 Microsoft CEO Satya Nadella Live on TBPN

Turn any episode into a week of clips.

This entire site, over 500 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

pre-training

Everything said about pre-training, oldest first

Turn any episode into a week of clips.