The Exchanges

Q the brain works is you're constantly predicting what you're going to see next. Like this may be why we see ghosts, but either way, your brain's constantly looking at what it thinks it expects to see. And then if something is not unexpected, it kind of jumps out at you. Is there anything like that with transformers or there's nothing, there's nothing like that where there's trying to predict things?

A Well, that's exactly how you train a model like GPT-IV. So, um, you know, if you take it to, to train a model like GPT-IV, basically we take all of the text, everything we can scrape off the internet. Um, and what the model is trying to do is it goes character by character and it's trying to predict the next character. So, you know, uh, if, if it sees, you know, the rain in Spain falls mainly on the, it's going to guess, oh, that's a P and it's going to be plain. And it does this over and over again for huge amounts of documents, like trillions of characters. And over time, it seems as though that yields something that looks like intelligence.

AI assessment note: “Well, that's exactly how you train a model like GPT-IV.”

Q Very interesting. And in, and in terms of, in terms of research projects going on, if anything you could talk about, is there anything you're really excited about for the next few years that's going to change things or any kind of new use cases that you're really excited about?

A Well, I, I think the, the thing that we have done that I am most excited about right now is, uh, something called code interpreter, uh, which is launched in beta right now. And it's, it's exactly what we were talking about. It's the ability for the model to write code and then run that code in a sandbox environment. And so you can do really cool things with it. You can give it a, you can upload a spreadsheet and you can have it analyze the spreadsheet for you. Um, you can have it make a, uh, you know, uh, animated GIF of, uh, you know, a data visualization. Um, and, and, you know, we're not specifically telling it like, okay, let's, let's make the spreadsheet model. Let's, let's give it this. It's just learn these things organically from everything on the internet.

AI assessment note: “thing that we have done that I am most excited about right now is, uh, something called code interpreter”

Q You might have asked it wrong. You might have misunderstood what, yeah, what it's doing. You might have misunderstood it. No, that's true. So tell us about the launch of ChatGPT, Bob. There's all sorts of competition going on there. What was, what was the, what was the scenario?

A Yeah, well, the funny thing is when we launched it, there was no competition. Um, so we had, we had launched GPT-III, uh, two years prior, and then we launched a better model, which is still, still okay, called GPT-III. And at the time, Um, the, the idea was we'd put these in an API. They weren't really ready for humans to talk to them yet. You know, if you, it, it'll finish your sentence, but it wouldn't have a conversation with you. And, um, we'd had these models out there for a while, and we had already then secretly trained GPT-IV. So this was the first half of last year. And GPT-IV was amazing. We knew that if we could, you know, figure out what to do with GPT-IV, it would, it would change everything. And so the whole company was focused on GPT-IV. What could we do with GPT-IV? And one guy, John Schulman, Uh, who ran our reinforcement learning team, said, you know what? What if we just make the models conversational? So we took one of the old models, GPT 3.5, trained it to be able to have a conversation. And we all thought the, the right ultimate path for the GPT models was for it to be an assistant to help you do things. But we were like, these models are clearly not good enough. We've had them out there for forever. And John was like, look, it's not perfect. I know GPT four is going to be the answer to everything, but let's just, let's just put it out on the internet and…

AI assessment note: “the funny thing is when we launched it, there was no competition.”

Q And for some of our listeners who aren't just technical, We're talking about a neural network. That's like, that's something that gives, that gives iterative feedback on things. Explain, how would you explain neural networks?

A So a neural network, um, the, the analogy that most people use is that it's like the brain, and it is, but at a very, very high level. So a neuron is basically, um, you can think of it as, as having connections to other neurons, and the strength of those connections determines when, you know, the first neuron, the lower level neuron fires, then that makes the, the higher level neuron fire. And if you train these, Um, you do it by showing them the right answer, and then basically doing what's called back-propagating the error, you know, from sort of the top of the network where the answer is all the way back down, um, through the whole network.

AI assessment note: “the analogy that most people use is that it's like the brain”

Q everyone always assumes there's like an exponential. It just goes to the moon and the world's over, you know, a singularity. And I always assume there's like this and then another S and then another S. Is it possible that there's like different paradigms you have to figure out that could take a while or do you think you guys have it all figured out enough to, to go away?

A I think there really are. I think, I think the, you know, these are, these things are fractal, right? And so in some sense, the paradigm is neural networks. But in another sense, well, you needed to figure out transformers. And so each of these S curves that, that this is a thing that genuinely happens. Each one is some new way of making sure that you can fit more compute and more data into a larger network. And, you know, every time you sort of see a peaking, you need to have some sort of breakthrough, but it's not as fundamental a breakthrough as it was before. Like the difference between neural networks and linear regression was really big. And the difference between neural networks now and neural networks two years ago is just a sequence series of tricks.

AI assessment note: “I think there really are. I think, I think the, you know, these are, these things are fractal”

Q Interesting. And, and, and did it end up crushing Dota too?

A Yeah, so we ended up beating Duda too, and I think what, what, for me, what was really crazy about this is that when I wanted to open AI, you know, Ilya Suskova, our chief scientist, sat me down and was like, we're gonna build AGI, it's gonna be human level intelligence, we're gonna do this in, you know, 1015 years, and I'm like, that would be cool if it were true, but I have no idea how we're gonna get there, you know, and I would ask him and he'd say, well, you know, we're gonna figure it out, and um, I remember one week I was sitting there, I had spent months You know, I was working on robotics, not on the Dota II problem, and I was trying to get a robot claw, like just a two-finger claw to grab, you know, a ball.

AI assessment note: “Yeah, so we ended up beating Duda too”