Brooks: OpenAI built Sora from scratch to achieve minute-long HD video
Tim Brooks · No Priors Ep.61 | OpenAI's Sora Leaders Aditya Ramesh, Tim Brooks and Bill Peebles · Apr 25, 2024 · at 13:58
Tim Brooks, co-lead of OpenAI's Sora, explains why OpenAI rejected the prevailing research method of patching image generation models to produce short video clips.
0:00 / 0:21exact quote · 21.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We started from scratch, and we started with the question of, how are we going to do a minute of HD footage? And that was our goal. And when you have that goal, we knew that we couldn't just extend an image generator. We knew that in order to do a minute of HD footage, we needed something that was scalable, that broke down data into a really simple way so that we could use scalable models.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Tim Brooks
Brooks: Sora represents the 'GPT-1' stage of generative visual models
“I think where we are in the trajectory of Sora right now is like, this is the GPT-I of these, this new paradigm of visual models, and that we're really looking at the fundamental research into making these way better, making it a way better engine that can pow…”
Brooks: Sora learned 3D spatial understanding purely from raw 2D video
“We didn't explicitly bake three D information into it whatsoever. We just trained it on video data and it learned about three D because three D exists in those videos.”
Brooks: The best way to scale intelligence is simple data prediction
“What works really well as you increase scale is just predict data. And that's what we do with text. We just predict text. And that's exactly what we're doing with visual data with Sora, which is we're not making some complicated trying to figure out some new t…”
Brooks: OpenAI has no immediate timeline to launch a Sora product
“We don't currently have immediate plans or even a timeline for creating a product.”
Brooks: AI teams should build for three-year end-states, not incremental features
“There is this pressure to do things fast because AI is so fast, and the fastest thing to do is, oh, let's take what's working now and let's kind of, like, add on something to it, and that probably is, as you're saying, more general than just image to video but…”
Brooks: Generative video will birth entirely new media formats within years
“I do think that, yeah, maybe over the next couple years we'll see people starting to make, like, more and more films, but I think people will also find completely new ways to use these models that are just different from the current media that we're used to.”