Justin Johnson: Native 3D AI representations will outperform 2D video generation
Justin Johnson · “The Future of AI is Here” — Fei-Fei Li Unveils the Next Frontier of AI · Sep 20, 2024 · at 31:00
Justin Johnson, co-founder of World Labs, explains why spatial AI models require underlying 3D world representations rather than relying on 2D video generation.
0:00 / 0:24exact quote · 24.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Modeling the two D projections of a dynamic three D world is, is a function that probably can be modeled, but by putting a three D representation into the heart of a model, there's just going to be a better fit between the kind of representation that the model is working on and the kind of tasks that you want that model to do. So our bet is that by threading a little bit more three D representation under the hood, That'll enable better affordances for users.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Justin Johnson
Justin Johnson: Seamless mixed reality will deprecate phones, TVs, and monitors
“If you've got the ability to seamlessly blend virtual content with the physical world, it kind of deprecates the need for all of those.”
Johnson: No base model has used new view prediction before Atlas
“And this is a really fundamental primitive that we think is super exciting, a super new primitive For base models that no one's ever done before, right?”
Johnson: Scaling spatial world models is primarily bottlenecked by training compute
“I think we're basically at the beginning, and we're basically limited by compute at this point, right? Like, data is very important, as Fei-Fei likes to point out, but, like, everything has a bottleneck, and I think the main bottleneck on continuing to scale t…”
Johnson: Generative new view prediction in Atlas is AI-complete
“But I think that's something we're kind of realizing, and Ben was talking about this earlier today, is, like, new view prediction, this primitive that we have in Atlas, especially generative new view prediction, this is also AI complete.”
Justin Johnson: Multimodal LLMs shoehorn visual data into 1D token sequences
“And now the multimodal LLMs that we're seeing now, you kind of end up shoehorning the other modalities into this underlying representation of a one D sequence of tokens. Now when we move to spatial intelligence, it's kind of going the other way. Where we're sa…”