McGrew: OpenAI secretly trained GPT-4 in first half of 2022
Bob McGrew · Ep 66: Bob McGrew — the Superstar Palantir Alum Leading OpenAI's Transformative Research Projects · Joe Lonsdale · Aug 18, 2023 · at 37:50
Bob McGrew, VP of Research at OpenAI, explains the timeline and internal focus at OpenAI prior to the unexpected launch of ChatGPT.
0:00 / 0:04exact quote · 4.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We had already then secretly trained GPT-IV. So this was the first half of last year.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Bob McGrew
McGrew: There is no limit to neural network capability with enough compute and data
“If you could figure out how to pour enough compute and pour enough data into a neural network, there's really no limit to how, how good it could get.”
McGrew: OpenAI models are improving, contrary to user reports of degradation
“Ah, we think it's actually getting better over time that, you know, some of the reports are sort of confounding various different things.”
McGrew: AI scaling will keep working without hitting an asymptote
“The way I think about it is that we are still pretty far away from the level of scale that is the human brain. And so there's no reason for there to be an asymptote. I think it just keeps working.”
McGrew: Human-level multi-agent AI will create its own civilization
“When you push this to the level where AI is at the human level, I mean, they're basically going to be creating their own civilization.”
McGrew: GPT-5 will render current AI infrastructure solutions completely obsolete
“I'm always a little worried about the infrastructure work, because I think, you know, you're solving the problems as they exist today, and when GPD 4.5 and GPD five come out, you know, they're gonna have fundamentally different use cases and fundamentally diff…”
McGrew: We understand neural networks much better than the human brain
“I, at this point, I think we understand neural networks a lot better than we understand the brain. So whenever someone talks to me about the brain, I always think, well, what does a neural network do? And then probably actually the brain does that.”