next token prediction
2 statements across 2 episodes · 1 bullish · 0 bearish · 2 people on the record · first statement Aug 4, 2025 by Stefano Ermon · across every show →
Everything said about next token prediction, oldest first
Aug 4, 2025 neutral
Ermon: Adapting Pretrained Causal LLMs to Diffusion Models Is Difficult
“The challenge is that, yeah, the training objective is quite different because you are training based on denoising as opposed to next token prediction. Diffusion models are not causal and that is also kind of problematic. I mean, it's a big advantage of diffus…”
Apr 27, 2026 bullish
Ludwig: Probably everything in the world reduces to next-token prediction
“So the interesting thing is, I think probably everything in the world can actually be boiled down to like a next token prediction problem. And In any workflow, anything can be thought of almost as like there's this, the sequence of steps or the sequence of tra…”