Oriol Vinyals
Co-founder, Discovery Loop · 1 appearance on the record.
computed by AI from the episodes · how this works → · full disclaimer →
scientistfounderexecutiveinvestor@oriolvinyalsml ↗LinkedIn ↗research.google.com/pubs/OriolVinyals.html ↗Wikipedia ↗
Oriol Vinyals is the VP of Research and Deep Learning Team Lead at Google DeepMind, as well as the Technical Co-lead of the Gemini project. He previously led the AlphaStar team to create competitive AI agents for StarCraft.
21statements → 14claims → 6claims resolved → 100%fully supported → 3.43/5average certainty → 2.05/5average debate potential → ≈4.0/5argument clarity, estimated →
6 supported 0 partly supported 0 contradicted 8 not checkable as stated how the 14 claims stand · each chip opens the sources
8 predictions · 6 assertions · 1 opinion · 6 insights · every statement was checked. The predictions and assertions are the 14 claims: statements the public record can support or contradict. 6 are resolved, and 8 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.
The record, in short
What the tape says about how Oriol argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.
Their most notable supported claim
How they sound: not measured why? →
We measure speaking style by listening to the audio itself, and a fair number needs at least 2,000 words from one person on tape we have measured. There is too little of Oriol Vinyals on measured tape to publish a rate. This says nothing about how they speak.
Everything Oriol Vinyals said on No Priors that made the record, most notable first. Filter by type, assessment or year in the ledger →
Vinyals predicts pre-training compute will drop to ~50% as RL expands
“So to me, that balance feels correct, like some on pre-training, and here we, we're trying to learn every task. So certainly that's going to be, you know, let's say it can be as high as 50%, not as high as over 90 like today. And then the rest mostly on reinfo…”
Vinyals: Emergent self-correction will make model bootstrapping trivial in specific domains
“At some point, the model might have the capability, let's say, to self-correct. Let's call it self-correction. And as soon as it hits that capability, you can see how, wow, bootstrapping now is trivial. And of course, it's not going to be that blanket across a…”
Vinyals: AI will remain hybrid as general models overtaking science is far away
“Probably we can, and we might see more and more bootstrapping from Gemini and these models to then a specific solution that, that the model is going to be throw away, except for, you know, maybe cracking protein folding or, you know, figuring out nuclear fusio…”
Vinyals: General reward functions in math will advance AI self-improvement
“Depending how you attack the problem, it's definitely not at that end. And I think even because it's so hard to have a perfect reward when you interleave language, I think by, even by accident, the field will move forward as we get to discover these more gener…”
Vinyals: Achieving monolithic AGI matters less than capability distribution
“The contrarian view would be, I'm not sure it matters that, that we achieved AGI. I think it might not look like Hey, like, it's gonna be exactly like the cognitive task we can do, and then we reach parity. It's gonna be a distribution of things that these mod…”
Vinyals: Context Windows Will Grow 10x for Commodity and SOTA AI in 1-2 Years
“I expect, I mean, order of one, two years, you might see, wow, we went another order of magnitude from both state of the art and, of course, what's considered a commodity. So I'm pretty certain about this.”
Vinyals: AI models show reasoning ability but remain inconsistently brittle
“Reasoning capabilities of the models are there, but I don't think we've perfected sort of making the reasoning very crisp and accurate so that these models would not hallucinate or would not, you know, the model might solve an, you know, Olympia mathematical p…”
Vinyals: Wrapping LLMs in iterative reasoning programs accelerates accuracy
“To accelerate that, that sort of progress, you want to start sort of really exploring what's the reasoning the model has, and by making it more redundant, more logical, by iterating more on these kind of ideas, you could imagine generating a very small program…”
Vinyals: LLM bootstrapping works if verification is easier than solution generation
“If checking that something is correct is easier than creating the solution, then we're in business because the language models will be able to evaluate their own samples more accurately than to generate them. And then we have a sort of reinforcement learning l…”
Vinyals: Gemini Reasons Over Entire Books Rather Than Compressing to Single Vectors
“The problem with, of course, retrieval based methods is they tend to simplify things to say, hey, like this whole book is just a single vector. Whereas if you just upload a whole book into Gemini and ask questions, it can really reason about every single word,…”
Vinyals: Future AI Research Will Drive Toward Hybrid Long-Context Retrieval Systems
“I think to me, it seems like a feature not a bug that we do have a bit of a hybrid mode perhaps going into the future and research will be driven like this.”
Vinyals: Language models inherently retain non-zero error probabilities
“So then you, of course, are absorbing all the knowledge on the internet and then sharpening those models around being, following instructions, being aligned with humans, but you still have this Probability distribution that will assign non-zero probability to …”
Vinyals: AlphaGo compute was mostly RL self-play, unlike modern LLM pre-training dominance
“Historically, if you look at AlphaGo, which actually followed quite closely the recipe of you pre-train your model on all human data, you then use RL to make it better, and then you do some search at inference time, the compute there was very skewed for the mi…”
Vinyals: General AI models are roughly 20% good at everything
“One way I kind of characterize general models is they are, I mean,
The level of performance is irrelevant, but they're like, 20% good at everything, ok?
So they, that's the level of performance they reach, but it's general, which, I mean, it's powerful, so you…”
Vinyals: Domain fine-tuning creates a synergistic loopback into general AI models
“That directionality of taking a generalist model and doing something by fine tuning it, there's going to be a loopback as well, right?
Then you're going to do something amazing, let's say in math, right?
We recently did that.
And then while the data or as a re…”
Vinyals: Reaching industry consensus on when AGI arrives will be impossible
“And honestly, it's gonna be also impossible to get agreement, so it's gonna be quite, Cloud of moments that people might feel it has happened now, but it doesn't matter because the models might be used for amazing things and products and research itself, right…”
Vinyals: Deep learning has cracked weather modeling, but not climate modeling
“Weather modeling is kind of cracked with deep learning, but climate is quite different.”
Vinyals: LLM research has at least 5 to 10 years of runway
“I think there's quite a lot of LLM research and related to be done for, I mean, five, 10 years at least.”
Vinyals: Gemini formed by merging parallel LLM efforts across Brain and DeepMind
“One was that the Gemini project was formed as a result of having two sort of Parallel efforts on LLMs, mostly led by Google Brain and what we now call Legacy DeepMind. So earlier in the year, there was an effort to merge the two projects, and that's when sort …”
Vinyals: RNNs and LSTMs Never Remembered Beyond a Few Hundred Words
“We come from a world where we had recurrent neural networks and LSTMs that actually had infinite memory, although it was not very capable, right? You, the models in, in practice, they never remember more than a few hundred words or so.”
Appearances (1)
| Episode | Date | Speaking time |
|---|---|---|
| No Priors Ep. 74 | With Google DeepMind VP of Research Oriol Vinyals | Aug 1, 2024 | 36m |