Vinyals: Domain fine-tuning creates a synergistic loopback into general AI models
Oriol Vinyals · No Priors Ep. 74 | With Google DeepMind VP of Research Oriol Vinyals · Aug 1, 2024 · at 34:31
Oriol Vinyals, VP of Research at Google DeepMind, explains how specialized AI models (such as math problem solvers) feed data back to improve general models like Gemini.
0:00 / 0:24exact quote · 24.9s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“That directionality of taking a generalist model and doing something by fine tuning it, there's going to be a loopback as well, right? Then you're going to do something amazing, let's say in math, right? We recently did that. And then while the data or as a reward model or something that model will do, Will help the main model get better as well. So it's quite a synergistic thing to do as well.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Oriol Vinyals
Vinyals predicts pre-training compute will drop to ~50% as RL expands
“So to me, that balance feels correct, like some on pre-training, and here we, we're trying to learn every task. So certainly that's going to be, you know, let's say it can be as high as 50%, not as high as over 90 like today. And then the rest mostly on reinfo…”
Vinyals: Emergent self-correction will make model bootstrapping trivial in specific domains
“At some point, the model might have the capability, let's say, to self-correct. Let's call it self-correction. And as soon as it hits that capability, you can see how, wow, bootstrapping now is trivial. And of course, it's not going to be that blanket across a…”
Vinyals: AI will remain hybrid as general models overtaking science is far away
“Probably we can, and we might see more and more bootstrapping from Gemini and these models to then a specific solution that, that the model is going to be throw away, except for, you know, maybe cracking protein folding or, you know, figuring out nuclear fusio…”
Vinyals: General reward functions in math will advance AI self-improvement
“Depending how you attack the problem, it's definitely not at that end. And I think even because it's so hard to have a perfect reward when you interleave language, I think by, even by accident, the field will move forward as we get to discover these more gener…”
Vinyals: Achieving monolithic AGI matters less than capability distribution
“The contrarian view would be, I'm not sure it matters that, that we achieved AGI. I think it might not look like Hey, like, it's gonna be exactly like the cognitive task we can do, and then we reach parity. It's gonna be a distribution of things that these mod…”