Vinyals predicts pre-training compute will drop to ~50% as RL expands
Oriol Vinyals · No Priors Ep. 74 | With Google DeepMind VP of Research Oriol Vinyals · Aug 1, 2024 · at 25:29
Google DeepMind VP of Research Oriol Vinyals discusses how AI compute will rebalance between pre-training, reinforcement learning, and inference.
0:00 / 0:26exact quote · 26.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So to me, that balance feels correct, like some on pre-training, and here we, we're trying to learn every task. So certainly that's going to be, you know, let's say it can be as high as 50%, not as high as over 90 like today. And then the rest mostly on reinforcement learning or if we get access to good rewards, that would be kind of the next Maybe piece of compute, but much bigger than it is today. And then inference time.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Oriol Vinyals
Vinyals: Emergent self-correction will make model bootstrapping trivial in specific domains
“At some point, the model might have the capability, let's say, to self-correct. Let's call it self-correction. And as soon as it hits that capability, you can see how, wow, bootstrapping now is trivial. And of course, it's not going to be that blanket across a…”
Vinyals: AI will remain hybrid as general models overtaking science is far away
“Probably we can, and we might see more and more bootstrapping from Gemini and these models to then a specific solution that, that the model is going to be throw away, except for, you know, maybe cracking protein folding or, you know, figuring out nuclear fusio…”
Vinyals: General reward functions in math will advance AI self-improvement
“Depending how you attack the problem, it's definitely not at that end. And I think even because it's so hard to have a perfect reward when you interleave language, I think by, even by accident, the field will move forward as we get to discover these more gener…”
Vinyals: Achieving monolithic AGI matters less than capability distribution
“The contrarian view would be, I'm not sure it matters that, that we achieved AGI. I think it might not look like Hey, like, it's gonna be exactly like the cognitive task we can do, and then we reach parity. It's gonna be a distribution of things that these mod…”
Vinyals: Context Windows Will Grow 10x for Commodity and SOTA AI in 1-2 Years
“I expect, I mean, order of one, two years, you might see, wow, we went another order of magnitude from both state of the art and, of course, what's considered a commodity. So I'm pretty certain about this.”
Vinyals: AI models show reasoning ability but remain inconsistently brittle
“Reasoning capabilities of the models are there, but I don't think we've perfected sort of making the reasoning very crisp and accurate so that these models would not hallucinate or would not, you know, the model might solve an, you know, Olympia mathematical p…”