pre-training paradigm

2 statements across 1 episodes · 1 bullish · 1 bearish · 1 people on the record · first statement Mar 28, 2025 by Dan Hendrycks · across every show →

Everything said about pre-training paradigm, oldest first

Mar 28, 2025 bullish

Assertion Not checkable as stated

Hendrycks: RL-based reasoning models are improving faster than pre-training did

“That is separate from the new reasoning paradigm that has emerged in the past year which is where you train models to on math and coding types of questions with reinforcement learning, and that has a very steep slope, and I don't see any signs of that slowing …”

Dan Hendrycks Mar 28, 2025 ▶ 24:15 AI's Rising Risks: Hacking, Virology, Loss of Control — With Dan Hendrycks

Mar 28, 2025 bearish

Opinion

Hendrycks: The next-token prediction paradigm in AI is running out of steam

“That sort of paradigm does seem like it's running out of steam. It has held for many, many orders of magnitude but the returns on doing that are lower.”

Dan Hendrycks Mar 28, 2025 ▶ 24:03 AI's Rising Risks: Hacking, Virology, Loss of Control — With Dan Hendrycks

Turn any episode into a week of clips.

This entire site, over 300 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

pre-training paradigm

Everything said about pre-training paradigm, oldest first

Turn any episode into a week of clips.