Ermon: Adapting Pretrained Causal LLMs to Diffusion Models Is Difficult
Stefano Ermon · ⚡️Mercury: Ultra-Fast Diffusion LLMs — Estefano Ermon, CEO Inception Labs · Aug 4, 2025 · at 7:29
Stefano Ermon, Stanford professor and Inception Labs CEO, explains why open-source autoregressive LLM checkpoints cannot be easily repurposed for discrete diffusion language models.
0:00 / 0:36exact quote · 36.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The challenge is that, yeah, the training objective is quite different because you are training based on denoising as opposed to next token prediction. Diffusion models are not causal and that is also kind of problematic. I mean, it's a big advantage of diffusion models that you don't have to be causal. You can actually look at the whole context to the left and to the right. As you decide, kind of like the edits that you want to do, which is giving, you know, it's a very powerful thing if you think about how you generate objects, but that makes it quite different, quite difficult to adapt kind of like models that have been trained with causal masking to this new task.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Stefano Ermon
Ermon: Diffusion models could become the dominant architecture over autoregressive models
“I'm pretty optimistic about a future where diffusion models
Can become the dominant solution. I've seen it happen before with GANs a few years ago, so I wouldn't be surprised if that's the case also here.”
Ermon: Diffusion LLMs Pareto-dominate autoregressive models on inference efficiency
“On the inference side, what we're seeing is that diffusion models are much more efficient. We're actually able to Pareto dominate autoregressive models. If you think about the typical trade-off between throughput versus latency, which you kind of like cannot, …”
Inception generalist model matches Claude Haiku quality at 5-10x speed
“We had our generalist model evaluated by artificial analysis and the intelligence score from AA artificial analysis around 40. So it's comparable to GPT, 4.1 nano, cloud haiku, kind of like Close source speed optimized models. It's roughly comparable in terms …”
Ermon: Power constraints will drive diffusion models to replace frontier LLMs
“If it happens, it's gonna be driven by efficiency. Like we're all constrained by essentially power. And if you have, I mean, at the end of the day, it's all an inference game, right? Okay. Training is expensive, but then the thing that matters is being able to…”