Interleaved multimodal generation overcomes single-shot limits via step-by-step planning
Mostafa Dehghani · AI is Already Building AI — Google DeepMind’s Mostafa Dehghani · Apr 2, 2026 · at 51:37
Mostafa Dehghani, AI research scientist at Google DeepMind, details how interleaved text-and-image generation in Gemini Flash Image (Nano Banana) changes the paradigm from simple text translation to image planning.
0:00 / 0:51exact quote · 51.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“But if you have incremental generation, so if you have text and then an image and text and image, you can get your model to generate these details one by one. So you never expect your model to generate an image, a perfect image in the first shot, right? So, so you expect model, your model to plan about this generation. So it says that, oh, you know, Let me start with big objects because, you know, later I'm going to have a hard time if I put like small objects and the big objects don't fit. Right. So let me just do that. And then like in the next turn, I go with like medium objects and smaller and this like super smart, you know, and you're never bothered by the capability of a single shot image generation because you did planning and then you tune every step difficulty to match the capability of your model to generate a one shot.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Mostafa Dehghani
New pre-training techniques will drastically boost base AI model capabilities
“The way that we used to do pre-training, maybe, like, you know, like two, a year ago or two years ago maybe, like, you know, diminishing return is, like, obvious, but I can see how new ideas are bringing, like, you know, fresh, fresh energy into the pre-traini…”
Fully automated AI self-improvement will eliminate human bottlenecks and trigger breakthroughs
“The moment that we had this full automation, I would say we can close the loop of self-improvement and then it becomes the Like, you know, the problems become like, you know, mostly providing compute for these models to actually do what they want to do. And as…”
AI model progress will alternate between pre-training and post-training breakthroughs
“We're going to be having a bit of a swing back and forth between pre-training and post-training.”
Post-training techniques cannot compensate for a weak base AI model
“Pre-training is still the foundation and like, you can never post-train your way out of a week-based model.”
Video data conveys physical world knowledge to AI more efficiently than text
“So because of that, like picking up a lot of knowledge about the word through language is just not really efficient. I don't want to say that it's impossible, but it's not efficient, you know, like to learn about gravity. If you kind of like, you know, have yo…”
Demonstrating that image training lowers text perplexity remains extremely difficult
“So it turned out to be a really, really good model, but it was like really hard to see that. Wow. You know, I train on images and then like Text perplexity goes down. That was hard to see. You know, like the fact that, you know, you train in native model and i…”