OpenAI VP of Research Bob McGrew discusses the long-term potential of scaling autonomous multi-agent AI systems with Joe Lonsdale.
Insight
McGrew: There is no limit to neural network capability with enough compute and data
“If you could figure out how to pour enough compute and pour enough data into a neural network, there's really no limit to how, how good it could get.”
Assertion Not checkable as stated
McGrew: OpenAI models are improving, contrary to user reports of degradation
“Ah, we think it's actually getting better over time that, you know, some of the reports are sort of confounding various different things.”
Prediction Not checkable as stated
McGrew: AI scaling will keep working without hitting an asymptote
“The way I think about it is that we are still pretty far away from the level of scale that is the human brain. And so there's no reason for there to be an asymptote. I think it just keeps working.”
Prediction Not checkable as stated
McGrew: GPT-5 will render current AI infrastructure solutions completely obsolete
“I'm always a little worried about the infrastructure work, because I think, you know, you're solving the problems as they exist today, and when GPD 4.5 and GPD five come out, you know, they're gonna have fundamentally different use cases and fundamentally diff…”
Opinion
McGrew: We understand neural networks much better than the human brain
“I, at this point, I think we understand neural networks a lot better than we understand the brain. So whenever someone talks to me about the brain, I always think, well, what does a neural network do? And then probably actually the brain does that.”
Insight
McGrew: GPT-3 and GPT-4 must be prompted to act smart or dumb
“Well, that's actually one of the cool things that we've discovered with GPT-III and GPT-IV, Is that the model, so the models are trained on the internet and they're reading everything, right? Then they try to act like what they've seen, right? They're predicti…”