Causal Masking
topic on 1 show · 1 statements across 1 episodes
1 statements about Causal Masking, every show
Ermon: Adapting Pretrained Causal LLMs to Diffusion Models Is Difficult
“The challenge is that, yeah, the training objective is quite different because you are training based on denoising as opposed to next token prediction. Diffusion models are not causal and that is also kind of problematic. I mean, it's a big advantage of diffus…”