Rombach: Stability AI distillation research enables single-step diffusion generation
Robin Rombach · Text to Video: The Next Leap in AI Generation · Feb 17, 2024 · at 7:05
Robin Rombach, lead AI researcher at Stability AI, discusses distillation techniques that dramatically reduce the number of sampling steps required by diffusion models.
0:00 / 0:12exact quote · 12.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We ourselves, we have published a distillation work a week ago that actually shows that you can go as low as one sampling step, which is, I would say like a big advantage of these diffusion models.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Robin Rombach
Rombach: Compute constraints drive AI innovation more than scaling hardware
“If you only rely on like more compute it's a bit boring. I think like compute constraints can also Drive innovation, right? So for example, the latent diffusion framework, we developed it at the university because we just, like, we had, like, single GPUs where…”
Rombach: Video AI models scaled like LLMs will gain world understanding
“Having something like we are seeing in language modeling, but trained on pixels on videos will probably give like super interesting downstream behavior to not, not only like generating videos, but also understanding of the world.”
Rombach: Pre-trained video models learn 3D synthesis faster than image models
“We showed that it's actually like helpful to incorporate like this implicit three D knowledge that Knowledge that is captured in all of the videos into the model, and then the model can learn much quicker than if you start from the pure image model.”
Rombach: AI video generation will evolve into real-time interactive experiences
“Because then this will become more like, I don't know, sometimes I think about this as like a video game, right? You type your prompt, and you immediately see what happens given your input view, and I think this might be a super nice user experience, actually.”