The Exchanges

Q And so I wanted to hear from each of you, like an all-star Mr. and Mrs. Like, what is Brad amazing at that the world doesn't know?

A Look, I think one of the, one sign of like a good partnership, I'm thankful to have this with like a lot of the key people at OpenAI, certainly Brad is like, um, if you can't do each other's job, Maybe Brad could do my job for a week. I certainly could not do Brad's job for a week. Um, and I, I think that ability to divide up as a team, um, and have a very high bandwidth communication channel with each person and all together as a leadership team is super important. Brad is good at a lot of things. Uh, I'll talk about just two here in the interest of time. One is adaptability. Uh, Brad joined, uh, To do finance, obviously, and now does something, I guess it's like in the sphere of finance, but very, very different. Um, we didn't have a business at all, or we didn't have an appreciable business until very recently. And when it became clear that we were going to have a very fast growing business, um, kind of like looked around, I was like, I really need somebody. We gotta, we gotta get someone to do this. And, uh, I kind of like looked around the room and I asked Brad to do it. And it was, and he was just like, okay, I'll figure it out. Like, I'll, you know, Uh, just like, you know, I might need, like, a little bit of time to get up to speed, but this is, you know, I've done, like, business-ish stuff before and can go, like, build all this out. So the willingness to just, like, t…

AI assessment note: “Brad is good at a lot of things. Uh, I'll talk about just two”

Q What do you think is the biggest barrier to that happening?

A I think the models are just not smart enough, which sounds like a annoying, low information kind of cop-out answer, but I think it's like deeply fundamentally true. Like the models just aren't smart enough. You fix that one thing, all these other things get better. There will be all these ways that we have to figure out how to integrate tools into people's workflow and, you know, modelability in different areas. Well, Will matter a lot, but if you zoom out, you know, doing scientific research with the help of GPT-II would have seemed fairly laughable. With GPT-IV, people do use it just in very, to help them do science, just in extremely primitive and limited ways. And with GPT-VI, I think people will say, hey, this is like helping me as a general purpose tool in all these ways. And then with GPT-VIII, maybe people are like, you know, this can do some limited, maybe not so limited tasks for me.

AI assessment note: “I think the models are just not smart enough”

Q What are the biggest things that would prevent or slow down the velocity of OpenAI's decision-making innovation?

A I think we have the best researchers and best research culture that I'm aware of in the world. Um, if we lost either of those things, that would be really bad. Not having enough compute resources would be, uh, really bad. And I think, you know, we, we love doing cool research because scientific advancement is like the coolest, most exciting thing in the world, but really we're here to like do useful stuff for other people. And if we do the best research in the world and then we make it as efficient as we can, but we still don't have enough compute. To provide it to everybody on earth who wants to use it and is going to want to use it so much more as these models get way better. That would get in the way. That'd be really bad. So, uh, the second thing I was going to say for priority is, uh, is thinking about how we get enough compute to fulfill the demand of people who want to use these.

AI assessment note: “Not having enough compute resources would be, uh, really bad.”

Q Is it difficult to maintain morale when it is long and winding roads, when training runs can fail? How do you maintain morale in those times?

A You know, we have a lot of people here who are excited to build AGI, and that, that's a very motivating thing, and no one expects that to be easy and a straight line to success, but there's a famous quote from history. It's something like, I'm gonna get this totally wrong, but the spirit of it is like, I never pray and ask for God to be on my side. You know, I pray and hope to be on God's side, and there is something about betting on deep learning that feels like being on the side of the angels, and you kind of just, it eventually seems to work out, even though you hit some big stumbling blocks along the way, and so like a deep belief in that has been good for us.

AI assessment note: “a deep belief in that has been good for us”

Q saw, but Massa sit on stage and say we will have, I'm not going to do an accent because my accents are terrible, um, but there will be nine trillion dollars of value created every single year, which will offset the nine trillion dollar capex that he thought would be needed. I'm just intrigued. How did you think about that when you saw that? How do you reflect on that?

A I can't put it down to, like, any, I think, like, if we can get it right with an orders of magnitude, that's, that's good enough for now. There's clearly going to be a lot of capex spent and clearly a lot of value created. This happens with every other mega technological revolution of which this is clearly one. Um, but, ah, you know, like, next year will be a big push for us into these next generation systems. You talked about when there could be, like, a no-code software agent. I don't know how long that's going to take, but if we use that as an example and imagine forward towards it, think about what, think about how much economic value gets unlocked for the world if anybody can just describe, like, a whole company's worth of software that they want. This is a ways away, obviously, but when we get there and have it happen, um, think about how difficult and how expensive that is now. Think about how much value it creates if you Keep the same amount of value, but make it wildly more accessible and less expensive. That, that's really powerful. And I think we'll see many other examples like that. We, I mentioned earlier, like, healthcare and education, but those are two that are both, like, trillions of dollars of value to the world to get right if you, and if AI can really, really, truly enable this to happen in a different way than it has before. I don't think big numbers are t…

AI assessment note: “if we can get it right with an orders of magnitude, that's, that's good enough”

Q Can I ask, when there were those moments of doubt from everyone else, which there were across those years, what gave you the conviction to stick at it when, bluntly, very few others had that same confidence?

A It just seemed to us like it was gonna work, and we kept making progress. Like, we, it was not, it was, I wouldn't call, I would not call it blind faith, although there is some amount if you just, you know, you gotta believe you can do a hard thing, but it It felt really important to us to do this, that if we could do it, it would be, you know, hugely meaningful. Um, To the world in some way. And that it might work. Like we had an attack vector we believed in. We had, uh, and then we had continued data that the approach was working. Of course, the specifics took a long time to figure out. Uh, you know, we did not start off doing language models. Obviously we kind of knew that if we could keep doing things that we previously thought were impossible, that was somehow a good sign for progress. And we had this like fundamental Conviction on the approach and the attack vector at a very high level for a very long time. And the details took a long time to work out and many brilliant discoveries by our colleagues. There was never any doubt that AI would be a big deal if we could do it. So that's helpful. Like it's, it's gonna be really valuable. Um, the approach we got successively more confident in, although it did take some wandering in the jungle for a while or the desert, whatever that phrase is. Um, and then, you know, it's like, If you believe something with high conviction and e…

AI assessment note: “It just seemed to us like it was gonna work, and we kept making progress.”

Q We have a five year horizon for OpenAI and a 10 year. If you have a magic wand and can paint that scenario for the five year and the 10 year, can you paint that canvas for me for the five and 10 year?

A I mean, I can easily do it for, like, the next Two years, but if we are right and we start to make systems that are so good at, you know, for example, helping us with scientific advancement, Actually, I will just say, I think in five years, it looks like we have an unbelievably rapid rate of improvement in technology itself. You know, people are like, man, the AGI moment came and went, whatever, the, like, the, the pace of progress is, like, totally crazy, and we're discovering all this new stuff, both about AI research and also about all of the rest of science, and that feels like if we could sit here now and look at it, it would, Seem like it should be very crazy. And then the second part of the prediction is that society itself actually changes surprisingly little. An example of this would be that I think if you asked people five years ago if computers were going to pass the Turing test, they would say no, and then if you said, well, what if an oracle told you it was going to, they would say, well, it would somehow be like just this crazy, breathtaking change for society, and we did kind of satisfy the Turing test, roughly speaking, of course, and society didn't change that much. It just sort of went whooshing by. And that's kind of, ah, example of what I expect to keep happening, which is progress, scientific progress keeps going. Outperforming all expectations and society …

AI assessment note: “I think in five years, it looks like we have an unbelievably rapid rate of improvement”

Q know, wave to the internet bubble, uh, in terms of, you know, the excitement and the exuberance, and I think the thing that's different is the amount that people are spending. Larry Ellison said that it will cost a hundred billion dollars to enter the foundation model race as a starting point. Do you agree with that statement? And when you saw that, were you like, yeah, that makes sense.

A Uh, no, I think it will cost less than that, but there's an interesting point here, um, which is everybody likes to use previous examples of a technology revolution to talk about, to put a new one into more familiar context, and A, I think that's a bad habit on the whole, and, but I understand why people do it, and B, I think the ones people pick for Analogizing AI are particularly bad. So the internet was obviously quite different than AI, and you brought up this one thing about cost, and whether it costs, like, ten billion or a hundred billion or whatever to be competitive, it was very, like, one of the defining things about the internet revolution was it was actually really easy to get started. Now, another thing that cuts more towards the internet is mostly For many companies, this will just be like a continuation of the internet. It's just like someone else makes these AI models, and you get to use them to build all sorts of great stuff, and it's like a new primitive for building technology. But if you're trying to build the AI itself, that's pretty different. Another example people use is electricity, um, which I think doesn't make sense for a ton of reasons. The one I like the most, caveated by my earlier comment that I don't think people should be doing this, or trying to, like, Use these analogies too seriously is the transistor. It was a new discovery of physics. It h…

AI assessment note: “no, I think it will cost less than that”

Q What are the one or two things that you think are most important to you now then?

A There are a lot of AI orgs in the world that can, um, copy what other people do. Uh, like once you know something is possible, once you kind of know the rest shape of it, once you know that people want it, that's not so hard. Um, or it's like somewhat hard. It's really hard to figure out how to do something new for the first time. And to do that consistently, ah, over years, and hopefully, if we're lucky enough, over decades. Building a research org and a product org, and a whole company that puts these things out in the world, because we also innovate on business models and anything else. This culture of repeated innovation, so that we're not just making GPT-V amazingly great, but six, seven, eight, whatever we're gonna call those, we won't keep numbering them like that at that point. Um, making sure that we're set up to do that, Uh, from a thinking about where the researchers can take us, what that means for where the product's got to go, what that means for the whole company has to follow. Um, that's a big one.

AI assessment note: “This culture of repeated innovation, so that we're not just making GPT-V amazingly great”

Q saw, but Massa sit on stage and say we will have, I'm not going to do an accent because my accents are terrible, um, but there will be nine trillion dollars of value created every single year, which will offset the nine trillion dollar capex that he thought would be needed. I'm just intrigued. How did you think about that when you saw that? How do you reflect on that?

A I can't put it down to, like, any, I think, like, if we can get it right with an orders of magnitude, that's, that's good enough for now. There's clearly going to be a lot of capex spent and clearly a lot of value created. This happens with every other mega technological revolution of which this is clearly one. Um, but, ah, you know, like, next year will be a big push for us into these next generation systems. You talked about when there could be, like, a no-code software agent. I don't know how long that's going to take, but if we use that as an example and imagine forward towards it, think about what, think about how much economic value gets unlocked for the world if anybody can just describe, like, a whole company's worth of software that they want. This is a ways away, obviously, but when we get there and have it happen, um, think about how difficult and how expensive that is now. Think about how much value it creates if you Keep the same amount of value, but make it wildly more accessible and less expensive. That, that's really powerful. And I think we'll see many other examples like that. We, I mentioned earlier, like, healthcare and education, but those are two that are both, like, trillions of dollars of value to the world to get right if you, and if AI can really, really, truly enable this to happen in a different way than it has before. I don't think big numbers are t…

AI assessment note: “I don't think big numbers are the point, and they're also the debate”

Q Is it difficult to maintain morale when it is long and winding roads, when training runs can fail? How do you maintain morale in those times?

A You know, we have a lot of people here who are excited to build AGI, and that, that's a very motivating thing, and no one expects that to be easy and a straight line to success, but there's a famous quote from history. It's something like, I'm gonna get this totally wrong, but the spirit of it is like, I never pray and ask for God to be on my side. You know, I pray and hope to be on God's side, and there is something about betting on deep learning that feels like being on the side of the angels, and you kind of just, it eventually seems to work out, even though you hit some big stumbling blocks along the way, and so like a deep belief in that has been good for us.

AI assessment note: “a deep belief in that has been good for us”

Q You've had an incredible journey over the last few years through You know, unbelievable hyper growth. You say about writing a book there in retirement. If you reflect back on the 10 years of leadership change that you've undergone, how have you changed your leadership most significantly?

A I think the thing that has been most unusual for me about these last couple of years is just the rate at which Things have changed at a normal company. You get time to go from zero to a hundred million in revenue, a hundred million to a 1,000,000,001, billion to ten billion. You don't have to do that in like a two year period. And you don't have to like build the company. We had to research that, but we really didn't have a company in the sense of a traditional Silicon Valley startup. That's, you know, scaling and serving lots of customers or whatever, um, having to do that. So. Quickly, there was just like a lot of stuff that I was supposed to get more time to learn than I got.

AI assessment note: “a lot of stuff that I was supposed to get more time to learn”

Q You've had an incredible journey over the last few years through You know, unbelievable hyper growth. You say about writing a book there in retirement. If you reflect back on the 10 years of leadership change that you've undergone, how have you changed your leadership most significantly?

A I think the thing that has been most unusual for me about these last couple of years is just the rate at which Things have changed at a normal company. You get time to go from zero to a hundred million in revenue, a hundred million to a 1,000,000,001, billion to ten billion. You don't have to do that in like a two year period. And you don't have to like build the company. We had to research that, but we really didn't have a company in the sense of a traditional Silicon Valley startup. That's, you know, scaling and serving lots of customers or whatever, um, having to do that. So. Quickly, there was just like a lot of stuff that I was supposed to get more time to learn than I got.

AI assessment note: “a lot of stuff that I was supposed to get more time to learn”

Q How does open source and the rise of open source further enable that or impact that?

A There will be a place for open source models in the world. Some people want them. Um, some people will want managed services. Some people, a lot of people use both. I kind of think all of these are Details that are like quite interesting in some sense, but miss the bigger picture, which is we are in the midst of a legitimate and pretty big technological revolution where intelligence is going from this very limited thing, which is, you know, smart humans have it. But if you like want to do something that requires a lot of intelligence, you got to get a lot of smart people to do something. Like if you want to make a thing like open AI, you need a ton of smart people, a ton. If you think about everything in the stack, not just people who work at OpenAI, but the people who make chips and build data centers and all of that, to something where one person will be able to access abundant and very inexpensive intelligence to do just amazing things.

AI assessment note: “all of these are Details that are like quite interesting in some sense, but miss the bigger picture”

Q How does open source and the rise of open source further enable that or impact that?

A There will be a place for open source models in the world. Some people want them. Um, some people will want managed services. Some people, a lot of people use both. I kind of think all of these are Details that are like quite interesting in some sense, but miss the bigger picture, which is we are in the midst of a legitimate and pretty big technological revolution where intelligence is going from this very limited thing, which is, you know, smart humans have it. But if you like want to do something that requires a lot of intelligence, you got to get a lot of smart people to do something. Like if you want to make a thing like open AI, you need a ton of smart people, a ton. If you think about everything in the stack, not just people who work at OpenAI, but the people who make chips and build data centers and all of that, to something where one person will be able to access abundant and very inexpensive intelligence to do just amazing things.

AI assessment note: “miss the bigger picture, which is we are in the midst of a legitimate”

Q Going off schedule is one thing. Trying to tease that out might get me in real trouble. How does open AI make breakthroughs in terms of like core reasoning? Do we need to start pushing into reinforcement learning as a pathway or other new techniques aside from the transformer?

A I mean, there's two questions and there's how we do it. And then, you know, there's everyone's favorite question, which is what comes beyond the transformer. How we do it is like our special sauce. It's easy. It's really easy to copy something you know works, and one of the reasons that people don't talk about about why it's so easy is you have the conviction to know it's possible, and so after, after a research lab does something, even if you don't know exactly how they did it, it's, I won't say easy, but it's doable to go off and copy it, and you can see this in the replications of GPT-IV, and I'm sure you'll see this in replications of O-one. What is really hard and the thing that I'm most proud of about our culture is the repeated ability to go off and do something New and totally unproven. And a lot of organizations, no, I'm not talking about AI research, just generally, a lot of organizations talk about the ability to do this. There are very few that do, um, across any field. And in some sense, I think this is one of the most important inputs to human progress. So one of the, like, retirement things I fantasize about doing is writing a book of everything I've learned. About how to build an organization and a culture that does this thing, not the organization that just copies what everybody else has done. Because I think this is something that the world could have a lot mo…

AI assessment note: “How we do it is like our special sauce.”

Q Why do you think you're not an operator?

A I mean, I manifestly not. Like, I, I was, I was very happy. Well, I had a lot of fun being an investor. Um, it's, it's not a fulfilling, it was not a fulfilling job for me, um, but it's a very fun one, and I kind of like, you know, all of the, like, things that people say to make fun of investors are somewhat true, like, for a quality of life job, it's a great, great trade-off. Um, but, yeah, with no false humility, I'm just not an operator by nature. I'm happy to do it, because I, like, really love OpenAI, and I think AGI will be the most important thing I ever touch, but this is not my natural fit.

AI assessment note: “with no false humility, I'm just not an operator by nature.”