Diffusion Language Models
topic on 1 show · 4 statements across 2 episodes
4 statements about Diffusion Language Models, every show
Wang: Diffusion Outperforms Autoregressive Models on Unseen Cellular Tasks
“We find that switching from autoregressive training to division language models give a significant improvement over some of the harder tasks, particularly generalized to unseen tasks.”
Wang: Autoregressive Models Type Sequentially, Diffusion Models Iteratively Edit
“What's the difference between autoregressive training versus diffusion language models is that it kind of, you can think of autoregressive training as typing. There is, for example, I like coffee, you have to type I, and then like, and coffee. There's inherent…”
Inception Labs develops specialized DPO algorithm for diffusion language models
“We have a DPO algorithm specialized for diffusion language models.”
Ermon: Diffusion LLMs gain speed by modifying multiple tokens in parallel
“That's kind of like the reason diffusion, diffusion language models are much faster compared to autoregressive models. Is that each neural network evaluation doesn't just give you one token, like in the typical autoregressive world, but it's able to Output, es…”