Wang: Autoregressive Models Type Sequentially, Diffusion Models Iteratively Edit
Bo Wang · 🔬Causal Models Need Causal Data - Xaira’s X-Cell model (Bo Wang & Ci Chu) · Jul 21, 2026 · at 42:53
Bo Wang, Chief AI Scientist at Xaira Therapeutics, explains why Xaira uses diffusion language models instead of autoregressive next-token prediction for virtual cell foundation models.
0:00 / 0:44exact quote · 44.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“What's the difference between autoregressive training versus diffusion language models is that it kind of, you can think of autoregressive training as typing. There is, for example, I like coffee, you have to type I, and then like, and coffee. There's inherent orders. But diffusion language model, you can treat it as editing. You iteratively generate a sentence from a very vague, very rough sentence, and then you can iteratively refine it. So same thing with gene expressions. You can generate a very rough representation of the gene expressions, and then iteratively from noisy representation to more refined representations. So you kind of iteratively edit the gene expression predictions until it minimize the losses.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Bo Wang
Wang: X-Cell Is First to Predict Unseen Cell Line Perturbations
“One of the rewarding signals I receive after we develop Excel is that like it's a wow moment from biologists that this is the first time biologists actually find the model can predict exactly how these unseen cell lines kind of respond to different perturbatio…”
Wang: Virtual cell AI models will eventually replace physical cellular experiments
“Eventually we kind of, we can replace all the cellular experiments by simply running simulations on computer without even running the actual Experiments.”
Wang: Diffusion Outperforms Autoregressive Models on Unseen Cellular Tasks
“We find that switching from autoregressive training to division language models give a significant improvement over some of the harder tasks, particularly generalized to unseen tasks.”
Wang: Data Quality and Scale Matter More Than Architecture or Priors
“Overall, if we have to give an order, my order would be the quality among scale of the datasets, and then the architecture, and then the prior knowledge.”
Wang: Foundation models on causal data will beat linear baselines
“I believe that foundation model or other more complicated AI models that trend on the right data will outperform these linear models in harder tasks, particularly in generalization tasks.”
Wang: X-Cell Can Predict Combinatorial Gene Perturbations In Silico
“This is also why we incorporate PPI networks as the prior knowledge into our model. And although the model right now are trained on single gene perturbations, but once the model is trained, you can actually predict combinatorial perturbations just on the model…”