The Exchanges

Q I immediately think, and you may kind of, um, chafe at this, but I immediately think of the social network where they are drawing the algebraic equations on the windows, uh, and you see that in the early scenes. How do you capture that process iteration thinking in what is previously non-existent or non-captured data?

A This is the right question. Now, the way that I think about it is that all of this got unlocked by DeepMind here in London, actually, in 2016. When DeepMind created AlphaGo, right, the, this incredible achievement of beating, you know, the world's best, you know, human player in Go, they did it in the following way, and I'm oversimplifying it for kind of brevity, but they first trained a neural net on all of the Go games that they could find online that had been played. And the outcoming Go player that the model had become was kind of average. So what they did next is they said, hey, Go is deterministic. There's a win and a loss scenario. And while there's close to infinite possible moves at every turn, an extremely large number, which was the part why we couldn't computationally brute force it, we do have these outcomes, win or lose. It's deterministic. So they gave a model the Go game engine, and through the use of reinforcement learning, let it explore moves and play against itself. And the model was learning from when it was winning and when it was losing about which moves to pick at which turn of the game. And in the end, it created AlphaGo. Now the way to think about it, and by the way, future versions were entirely in simulated environments. They didn't require anymore the human played games to bootstrap it. And the reason what kind of DeepMind did is they saw a place wh…

AI assessment note: “we're in a simulatable domain, so we can simulate the data.”

Q it's valuable, you said, again, I'm jumping around the different notes, but you said about closing the gap, and specifically with regards to code, and we chatted earlier about that as with regards to other issues, I think we chatted about voice recognition as an alternative. How do you think about this element of closing the gap and how that correlates to where value is and maybe where it isn't?

A The way I think about this is there's things in the world that today we consider economically valuable. So if we take what's economically valuable, the next thing we need to ask ourselves What's the gap between models today and human level capabilities, and how large is that gap? And in some cases, the gap is actually not that large anymore. We were talking earlier about speech recognition. Models today, in my opinion, are pretty much there. Maybe there's a tiny bit left to say, but we've closed that gap, you know, into an incredible amount. Uh, in other areas, the gap felt like it was going to be impossible to close, but we're making a lot of progress. Come back to full self-driving. If you've been in your latest, you know, Tesla FSD update, that gap feels getting closer and closer to be closed. Now, there's other areas where the gap is really large still. I think software development, our domain, we think the gap is still very large, right? What models are able to do is they're massively useful assistance, and they drive massive economic value because of that. But between a model Working with a developer today, there's a huge, huge gap, and we want to get to a world where developers can work with models that are as capable as them, right, and potentially even one day more capable. Now, the reason I mentioned this is so we've got the human capability aspect, right? What's the …

AI assessment note: “What's the gap between models today and human level capabilities, and how large is that gap?”

Q Because of some of the egregious spending, Larry Ellison said on a stage recently, it will require a hundred billion dollars to enter the race. That is the entry price. It was a really striking moment for me where I was like, oh my god, I, I don't know what the future holds. Do you agree with that as an entry price?

A If you want to become Uh, a hyperscaler that is able to put data centers all over the world with GPUs in it that is, that are going to allow you to serve these models to everyone, an infrastructure player, that's probably it. And that's probably just a starting point, right? If we look at the, the massive CapEx investments that, uh, all of the, the cloud companies are doing, uh, you know, they're, they're far above a hundred billion dollars when you look at them over the course of, you know, a couple of years. Now, in the race towards More and more capable AI, closing that gap between human intelligence and, and machine intelligence, uh, I think we are all pushing the frontier more and more possible, and we're seeing how that gap closes as we're scaling up our models and scaling up our data. I don't think anyone has a definite answer of how many dollars is it going to take from here to there. If we knew that, we knew the outcomes. We're all on the frontier of what's possible right now.

AI assessment note: “an infrastructure player, that's probably it. And that's probably just a starting point”

Q Because of some of the egregious spending, Larry Ellison said on a stage recently, it will require a hundred billion dollars to enter the race. That is the entry price. It was a really striking moment for me where I was like, oh my god, I, I don't know what the future holds. Do you agree with that as an entry price?

A If you want to become Uh, a hyperscaler that is able to put data centers all over the world with GPUs in it that is, that are going to allow you to serve these models to everyone, an infrastructure player, that's probably it. And that's probably just a starting point, right? If we look at the, the massive CapEx investments that, uh, all of the, the cloud companies are doing, uh, you know, they're, they're far above a hundred billion dollars when you look at them over the course of, you know, a couple of years. Now, in the race towards More and more capable AI, closing that gap between human intelligence and, and machine intelligence, uh, I think we are all pushing the frontier more and more possible, and we're seeing how that gap closes as we're scaling up our models and scaling up our data. I don't think anyone has a definite answer of how many dollars is it going to take from here to there. If we knew that, we knew the outcomes. We're all on the frontier of what's possible right now.

AI assessment note: “If you want to become... an infrastructure player, that's probably it.”

Q it's valuable, you said, again, I'm jumping around the different notes, but you said about closing the gap, and specifically with regards to code, and we chatted earlier about that as with regards to other issues, I think we chatted about voice recognition as an alternative. How do you think about this element of closing the gap and how that correlates to where value is and maybe where it isn't?

A The way I think about this is there's things in the world that today we consider economically valuable. So if we take what's economically valuable, the next thing we need to ask ourselves What's the gap between models today and human level capabilities, and how large is that gap? And in some cases, the gap is actually not that large anymore. We were talking earlier about speech recognition. Models today, in my opinion, are pretty much there. Maybe there's a tiny bit left to say, but we've closed that gap, you know, into an incredible amount. Uh, in other areas, the gap felt like it was going to be impossible to close, but we're making a lot of progress. Come back to full self-driving. If you've been in your latest, you know, Tesla FSD update, that gap feels getting closer and closer to be closed. Now, there's other areas where the gap is really large still. I think software development, our domain, we think the gap is still very large, right? What models are able to do is they're massively useful assistance, and they drive massive economic value because of that. But between a model Working with a developer today, there's a huge, huge gap, and we want to get to a world where developers can work with models that are as capable as them, right, and potentially even one day more capable. Now, the reason I mentioned this is so we've got the human capability aspect, right? What's the …

AI assessment note: “take what's economically valuable, the next thing we need to ask ourselves What's the gap”

Q physical infrastructure, It was David Kahn. I'm trying to get exactly what he said, just because I don't want to butcher it, giving it someone else's quote, but he said that essentially you will never train a frontier model on the same data center twice, meaning that, you know, the evolution of models is now outpacing the development of data centers. Do you agree with him when you hear that?

A We're in a world today where the amount of data centers that can hold And power and have enough energy, uh, to power increasingly magnitude order largest of clusters, uh, is a very small number. Uh, and, and so I, I think he's absolutely right in this sense. The data centers from, you know, two years ago versus the data centers in terms of size and power requirement that we're going to see in the next two years look radically different, not just because the scale of number of servers and nodes that we're interconnecting. This is the difference between inference, right? For inference, We don't need all of the machines to be connected to each other in the same place. For training, we need them all to be connected to each other in the same room, in the same place. And so that massively changes what a data center looks like.

AI assessment note: “I think he's absolutely right in this sense.”

Q I'm sorry. I think the show has done so well because I ask questions that people think, why do you need that for training and not for inference?

A When we're scaling up the size of these models, and we're training them on more and more data, and we're using more and more compute for it, at every single step that we're taking, In the learning, every set of samples of data that we show the model, we need them to communicate with each other and share what they've learned across the optimization landscape. And so this is means that if I would, you know, have two data centers that sit far away from each other, the, the, the amount of information that they have to share with each other, all of the different servers, and we're talking here 1010 of thousands soon of servers. You know, would make it so slow that it wouldn't be economically viable to train these models. Once I'm running a model, I'm using a lot less servers to run it. So think of it as having lots and lots of copies of the model during training over lots and lots of machines, that every time they see data, they learn, they need to communicate with each other to continue to improve in their learning.

AI assessment note: “we need them to communicate with each other and share what they've learned”

Q On the synthetic data side, a lot of people use it as a capsule for like, oh, we've got a data shortage problem, but don't worry, synthetic data's here to save us. To what extent is all synthetic data equally valuable, or is it more valuable in certain industries versus others?

A I think the biggest cognitive dissonance that people have around synthetic data is a model is generating data to then actually become smarter itself. Right? It feels like the snake eating itself. There's something that doesn't make sense in it. Now, the way that you need to look at that is that there's actually another step in that loop. There's something that determines if from all the data that the model generated, so think of this as a, in my domain in software development, I have a task in a code base, and the model generates a hundred different solutions. Now, if I would just feed those hundred different solutions back to the model and its training, the model won't get smarter. That's the snake eating itself. But if you have something that can determine an oracle of truth that can help say, this is better and this is worse, or this is correct and this is wrong, that's when you can actually use synthetic data. Now in our case, that's executing the code. Did it actually run? Did it pass the tests? That gives me validation that the reasoning and the output are correct.

AI assessment note: “if you have something that can determine an oracle of truth... that's when you can”

Q Paris is meant to be the AI hub of Europe, no?

A Where has talent historically been, even pre-Chatsuppie team moment, right, and talent in AI? And who helped build that talent in this space? The number one company we have to give credit to is DeepMind. DeepMind built an incredible talent base, and they built it out of London. Meta did some work in building a very incredible talent base, and it did it between London and Paris. But in terms of when you look at it from a numbers perspective and sheer size of people, I think, you know, Google separately and DeepMind as part of Google had made much larger investments. And then there's another talent pool that we do often talk about publicly that is just absolutely extraordinary, which is Yandex. Yandex built an incredible company in Russia with some of the world's most capable researchers and engineers, many of which have since left Russia and have kind of become a diaspora all over Europe.

AI assessment note: “Meta did some work in building a very incredible talent base, and it did it between London and Paris.”

Q Would you have done poolside if you had sold sourced?

A I think the question is, what, what could I have been able to do continuing on sourced mission? Because sourced mission was The mission we're talking about today with Poolside. And back in 2016, there were very few people who believed it was ever possible for AI to write code. But no, I don't think, there's really no regrets there. I wouldn't be sitting where I am today, and I don't think I would have Become the person that allows me to go build poolside today. And frankly, I'm really grateful that that event did happen because that's how I met my co-founder. That's how I met Jason. He was the CTO at GitHub at the time, and it started this many-year conversation on what the progress in AI looks like and its applicability to software development.

AI assessment note: “I don't think I would have Become the person that allows me to go build poolside”

Q I'm sorry. I think the show has done so well because I ask questions that people think, why do you need that for training and not for inference?

A When we're scaling up the size of these models, and we're training them on more and more data, and we're using more and more compute for it, at every single step that we're taking, In the learning, every set of samples of data that we show the model, we need them to communicate with each other and share what they've learned across the optimization landscape. And so this is means that if I would, you know, have two data centers that sit far away from each other, the, the, the amount of information that they have to share with each other, all of the different servers, and we're talking here 1010 of thousands soon of servers. You know, would make it so slow that it wouldn't be economically viable to train these models. Once I'm running a model, I'm using a lot less servers to run it. So think of it as having lots and lots of copies of the model during training over lots and lots of machines, that every time they see data, they learn, they need to communicate with each other to continue to improve in their learning.

AI assessment note: “Once I'm running a model, I'm using a lot less servers to run it.”

Q talent before being such a crucial part that we haven't really unpacked because we have discussed the models, the data, the compute. The talent perspective is one that you also have taken quite a different approach on. You know, you're a European-based company. The big question that a lot of your investors said that we have to discuss is, why did you decide to keep this as a European-based company?

A I want to set the record straight. Uh, we're an American company, and we've got incredible people from all the way from San Francisco to Israel. A decision that we made early on is we were actually, Jason and I, my co-friend and I, we were planning on building this company in the Bay Area, and we did the work in the first days of the company, and the work was, let's make a list of everyone we think from both that we knew and also, like, externally, like, you know, on research papers and, and, and GitHub repos that we think potentially could be great for us. And the list ended up with about 3300 people. A lot of work done, and...

AI assessment note: “I want to set the record straight. Uh, we're an American company”

Q I want to kind of unpack that kind of one by one. If we start, I mentioned the algos, I mentioned the data, I mentioned the compute. You said about kind of algos and how it approves model efficiency. Is there a limit to how efficient models can and will get? And does that kind of plateau at some point?

A We are horribly inefficient at learning today. If you think about what drives efficiency of learning, It's the algorithms and it's the, and it's the hardware itself. And we've got probably decades, if not, you know, hundreds of years of, of improvements still left there and different forms of it over time. If we look very practically in the coming years, we are going to see increasing advantages on the hardware. I'm going to see increasing advantages on the algorithms, but I hope everyone takes away that this is table stakes. This is something that you have to do to be in this space and you have to be excellent at it. It's not what differentiates you, it's what allows you to keep up with everyone else.

AI assessment note: “we've got probably decades, if not, you know, hundreds of years of, of improvements”

Q Does cash correlate to compute? And what I mean by that is, if you have cash, Can you go to your store and say, I want this amount of compute, please? Or is it more than that?

A It's the right question. I think, again, it depends on how much cash and how much compute. About a year and a half ago when we started as a company, there was a true imbalance between supply and demand in the world that even as a, as you know, frontier AI company starting this, everyone wants you to win. NVIDIA is incentivized to hyperscale. Everyone is incentivized actually to make early stage companies succeed We compute. It's a lot easier when you're an early stage AI company or frontier AI company in general to get compute than it is when you're an enterprise, because they understand this is where the future is heading towards. But even then, there was a real mismatch between demand and supply, and we had to do an incredible amount of work of, of understanding the market, building relationships, and having plan A to Z to get there. In the last six months, I think the world has still a huge supply shortage. And we can, and we can see this, but when you are, where if you're an early stage startup, there's lots of paths for you. If you're a frontier AI company, you need to make decisions about who do you partner with? Who do you work with? How much do you do yourself? You need to, I'm making decisions today that will impact us on compute in 12 or 18 months from now. It's very rare to be at early stage companies where you have to make decisions right now that impact you, you kn…

AI assessment note: “we had to do an incredible amount of work of, of understanding the market, building relationships”

Q talent before being such a crucial part that we haven't really unpacked because we have discussed the models, the data, the compute. The talent perspective is one that you also have taken quite a different approach on. You know, you're a European-based company. The big question that a lot of your investors said that we have to discuss is, why did you decide to keep this as a European-based company?

A I want to set the record straight. Uh, we're an American company, and we've got incredible people from all the way from San Francisco to Israel. A decision that we made early on is we were actually, Jason and I, my co-friend and I, we were planning on building this company in the Bay Area, and we did the work in the first days of the company, and the work was, let's make a list of everyone we think from both that we knew and also, like, externally, like, you know, on research papers and, and, and GitHub repos that we think potentially could be great for us. And the list ended up with about 3300 people. A lot of work done, and...

AI assessment note: “I want to set the record straight. Uh, we're an American company”

Q When we look at that talent, when we think about work ethic, it's one thing which Europe is often chastised for. In terms of work-life balance, how do you approach that and feel about implementing standards of work with teams?

A There was a tweet, and if I recall correctly, it's from Aaron, Levi from Box, early on in post-ChatGPT moment, and he wrote something along the lines of, if you feel like you're working extremely hard on reasonable hours as AI is now booming, you're probably right to do so, because it's in these first years that, and I'm probably going beyond what the tweet said, but I think it's in these first years it's where the table gets set. Who has earned the right to be in the race to AGI? And the way that I've, I've always looked at this is from a personal perspective, and so has my co-founder, and so has Margarita, like, has, has put us in a place where we're gonna look back on this moment 10 years from now, just like we would look back to the moment of mobile, internet, you know, first personal computer, and realize that that was the moment where the table got set. And you do not want to, you do not want to look back on that moment and not have given it everything you've got, because it's a race. And look, most startups are not racist. Most startups are against your, yourself. But AGI is a race, and so our view always has been, is that the team that we build is a team that is deeply passionate to be in that race, and, and knows that, and knows And frankly, when you decide to join a race and you're upfront about it, you decide to be, try to become the gold medalist in swimming, that m…

AI assessment note: “You don't get to have it all. And, and so that's something that we've been super open”

Q Last one, and then we'll do a quick fire. You mentioned BYD. Unbelievable journey. Are China really two years behind?

A No. No, they're not. Uh, there's a couple of interesting things that might not be as obvious unless you're in our industries. The research that still gets published openly, right, that doesn't get held back, that is most interesting, is all coming out of China in vast spades and majorities. Something that wouldn't necessarily be obvious, but if you think about the game theory optimal thing to do, because they're not on the forefront of the world scene of AI, actually opening up some of that research is the game theory optional, you know, optimal thing to do to be able to continue to attract talent. Because that's really what opening up your research does, right? It attracts talent to you. Uh, no, I think, I think China is at an incredible level of capabilities, uh, and in no way should be discarded or thought of as years behind, uh, on AI or AGI progress. We're working on technologies that we can see have massive societal impact. And I think it's really important to be good stewards of that technology and that progress. And I think part of that is acknowledging that what we know about is the technology. What we know about is our users, our customers. But we, we should be, you know, very much be careful in terms of trying to know what's best for the world and how to think about massive geopolitical conflicts and things like that. The best thing that we can do is the West. Is to …

AI assessment note: “No. No, they're not.”

Q On the synthetic data side, a lot of people use it as a capsule for like, oh, we've got a data shortage problem, but don't worry, synthetic data's here to save us. To what extent is all synthetic data equally valuable, or is it more valuable in certain industries versus others?

A I think the biggest cognitive dissonance that people have around synthetic data is a model is generating data to then actually become smarter itself. Right? It feels like the snake eating itself. There's something that doesn't make sense in it. Now, the way that you need to look at that is that there's actually another step in that loop. There's something that determines if from all the data that the model generated, so think of this as a, in my domain in software development, I have a task in a code base, and the model generates a hundred different solutions. Now, if I would just feed those hundred different solutions back to the model and its training, the model won't get smarter. That's the snake eating itself. But if you have something that can determine an oracle of truth that can help say, this is better and this is worse, or this is correct and this is wrong, that's when you can actually use synthetic data. Now in our case, that's executing the code. Did it actually run? Did it pass the tests? That gives me validation that the reasoning and the output are correct.

AI assessment note: “if you have something that can determine an oracle of truth that can help say”

Q Did you go through a phase of getting stuff?

A I did. And I was lucky. I went through it. Uh, in my early twenties. And a lot of stuff. And then I got rid of all of it. And, uh, and I think the, for me the sheer realization is that it's the, it's the journey with the people. The outcomes, you know, that we want to see in the world. There's a, they're part of the obsession. They're the, I, if you can see a future that looks like one that ends up being this incredible future, you want to build it. But that moment, once you reach that, which will always be an ever moving goal, is not the interesting moment. The interesting moment is every single day with the people. And I think for me, I just learned more and more over the years is that I, I love people and, and I love the people I work with on my team. I think in every single person, there's something incredible. And, and, If you cut all the stuff, you cut all the money, and, and you pick the hardest, biggest thing you could possibly, you know, focus your life on, and then do it with amazing people, you get to have this incredible experience.

AI assessment note: “I did. And I was lucky. I went through it. Uh, in my early twenties.”

Q before that. You mentioned it earlier. There's different opinions around this. A lot of people now have come to the conclusion that we haven't even touched the surface, and scaling laws have so much more room to play out, and others have a lot more negative views, bluntly. How do you feel about our, where we are in terms of scaling laws, and how much room we have to run?

A So I think we are starting to understand that the scaling, the first version of the scaling laws that came out spoke about The amount of data we provided during training, this, and the size of the model, right? And, and more data, longer training, and size of the model, larger requires more compute. And so we often say, hey, the scaling laws are about applying more compute. And that's actually more correct than we initially realized, because the importance of synthetic data for models to get better is another form of using compute. But we're using it at inference time. We're running these models to generate these hundred solutions, generate a thousand, or a hundred, or 50. I think we have a lot of room still for scaling up models. We can do this by scaling up data, and we can do this by scaling up the size of the model. And I do think we're Where in this case, not so different than I think most of the major, you know, other major AI companies in the space, is that there's a lot of room to scale the number of parameters and size of models still. But there's something that we don't really talk about in our industry as much. Well, Train is extremely large models. And by the way, we until very recently weren't even capable of doing so because we didn't have the compute and the capital. This is why our fundraise, you know, has been so important to us so that we can have the capital …

AI assessment note: “I think we have a lot of room still for scaling up models.”

Q before that. You mentioned it earlier. There's different opinions around this. A lot of people now have come to the conclusion that we haven't even touched the surface, and scaling laws have so much more room to play out, and others have a lot more negative views, bluntly. How do you feel about our, where we are in terms of scaling laws, and how much room we have to run?

A So I think we are starting to understand that the scaling, the first version of the scaling laws that came out spoke about The amount of data we provided during training, this, and the size of the model, right? And, and more data, longer training, and size of the model, larger requires more compute. And so we often say, hey, the scaling laws are about applying more compute. And that's actually more correct than we initially realized, because the importance of synthetic data for models to get better is another form of using compute. But we're using it at inference time. We're running these models to generate these hundred solutions, generate a thousand, or a hundred, or 50. I think we have a lot of room still for scaling up models. We can do this by scaling up data, and we can do this by scaling up the size of the model. And I do think we're Where in this case, not so different than I think most of the major, you know, other major AI companies in the space, is that there's a lot of room to scale the number of parameters and size of models still. But there's something that we don't really talk about in our industry as much. Well, Train is extremely large models. And by the way, we until very recently weren't even capable of doing so because we didn't have the compute and the capital. This is why our fundraise, you know, has been so important to us so that we can have the capital …

AI assessment note: “I think we have a lot of room still for scaling up models.”

Q I want to kind of unpack that kind of one by one. If we start, I mentioned the algos, I mentioned the data, I mentioned the compute. You said about kind of algos and how it approves model efficiency. Is there a limit to how efficient models can and will get? And does that kind of plateau at some point?

A We are horribly inefficient at learning today. If you think about what drives efficiency of learning, It's the algorithms and it's the, and it's the hardware itself. And we've got probably decades, if not, you know, hundreds of years of, of improvements still left there and different forms of it over time. If we look very practically in the coming years, we are going to see increasing advantages on the hardware. I'm going to see increasing advantages on the algorithms, but I hope everyone takes away that this is table stakes. This is something that you have to do to be in this space and you have to be excellent at it. It's not what differentiates you, it's what allows you to keep up with everyone else.

AI assessment note: “we've got probably decades, if not, you know, hundreds of years of improvements”

Q I immediately think, and you may kind of, um, chafe at this, but I immediately think of the social network where they are drawing the algebraic equations on the windows, uh, and you see that in the early scenes. How do you capture that process iteration thinking in what is previously non-existent or non-captured data?

A This is the right question. Now, the way that I think about it is that all of this got unlocked by DeepMind here in London, actually, in 2016. When DeepMind created AlphaGo, right, the, this incredible achievement of beating, you know, the world's best, you know, human player in Go, they did it in the following way, and I'm oversimplifying it for kind of brevity, but they first trained a neural net on all of the Go games that they could find online that had been played. And the outcoming Go player that the model had become was kind of average. So what they did next is they said, hey, Go is deterministic. There's a win and a loss scenario. And while there's close to infinite possible moves at every turn, an extremely large number, which was the part why we couldn't computationally brute force it, we do have these outcomes, win or lose. It's deterministic. So they gave a model the Go game engine, and through the use of reinforcement learning, let it explore moves and play against itself. And the model was learning from when it was winning and when it was losing about which moves to pick at which turn of the game. And in the end, it created AlphaGo. Now the way to think about it, and by the way, future versions were entirely in simulated environments. They didn't require anymore the human played games to bootstrap it. And the reason what kind of DeepMind did is they saw a place wh…

AI assessment note: “we're in a simulatable domain, so we can simulate the data”

Q My immediate thought jumps to GitHub. Are GitHub not the best place to do that?

A GitHub today has this incredible data set, which is all of the code, almost all of the code in the world. GitLab is a player, but only in the private side, right, what sits behind the accounts of developers. GitHub is massive in public code, and it's massive in private code. But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data. And it's the output data. And so there is an inherent advantage from a capabilities race perspective. And another thing that we frame in our company over and over again is there's a capabilities race in the world. And to your point earlier, we said there's four things that matter. I agree with you on the three, but I'm going to add one, right? It's compute. It's data, it's proprietary applied research, it's the algorithms, and talent. Talent is absolutely key in this industry. Now in the go-to-market race, it's talent first and foremost, but it's also product and distribution. In distribution, Microsoft definitely has an incredible positioning in the world.

AI assessment note: “So all of us have access to the same public data.”

Q A lot of people break it down as compute data and then kind of models themselves. Well, I mean, yeah, compute data and then algorithms, really. So if we take those three, how do you think about what the biggest bottleneck is today in the progression of models? Is it the data that we mentioned, or is it one of the other two?

A We are making, in our space, Especially, I think, post the ChatGPT moment, like incredible advancements in the algorithms that are making learning more efficient. Internally, I, I have this thing that I say to the team, and they're probably tired of me hearing because I say it every single day. I say, all the work we do on foundation models, on one hand, is improving their compute efficiency for training or running them, or on the other hand, improving data. Now, the way to think about the algorithms and the improvement of compute efficiency is that's table stakes. All of us, OpenAI, Anthropic, Google, et cetera, are doing this, and we're just constantly improving here. And it's engineering and research combined. But the real differentiation between two models is the data. But compute matters tremendously for data. Because if you think about poolside, and we spoke about how do we get this data, and I mentioned the word synthetic, it means that we're generating it. It means that we're using models to generate data. To then actually use models to evaluate it, to then run it. And so, compute usually matters on this side of the generation of data. But once we have all of this data, where we started there, we spoke about, you know, neural nets essentially being compression of data that forces and generalizes learning. Now, when we have small models, We are taking huge amounts of dat…

AI assessment note: “the real differentiation between two models is the data. But compute matters tremendously”

Q My immediate thought jumps to GitHub. Are GitHub not the best place to do that?

A GitHub today has this incredible data set, which is all of the code, almost all of the code in the world. GitLab is a player, but only in the private side, right, what sits behind the accounts of developers. GitHub is massive in public code, and it's massive in private code. But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data. And it's the output data. And so there is an inherent advantage from a capabilities race perspective. And another thing that we frame in our company over and over again is there's a capabilities race in the world. And to your point earlier, we said there's four things that matter. I agree with you on the three, but I'm going to add one, right? It's compute. It's data, it's proprietary applied research, it's the algorithms, and talent. Talent is absolutely key in this industry. Now in the go-to-market race, it's talent first and foremost, but it's also product and distribution. In distribution, Microsoft definitely has an incredible positioning in the world.

AI assessment note: “GitHub is massive in public code... But private code, no one's allowed to train on.”

Q Last one, and then we'll do a quick fire. You mentioned BYD. Unbelievable journey. Are China really two years behind?

A No. No, they're not. Uh, there's a couple of interesting things that might not be as obvious unless you're in our industries. The research that still gets published openly, right, that doesn't get held back, that is most interesting, is all coming out of China in vast spades and majorities. Something that wouldn't necessarily be obvious, but if you think about the game theory optimal thing to do, because they're not on the forefront of the world scene of AI, actually opening up some of that research is the game theory optional, you know, optimal thing to do to be able to continue to attract talent. Because that's really what opening up your research does, right? It attracts talent to you. Uh, no, I think, I think China is at an incredible level of capabilities, uh, and in no way should be discarded or thought of as years behind, uh, on AI or AGI progress. We're working on technologies that we can see have massive societal impact. And I think it's really important to be good stewards of that technology and that progress. And I think part of that is acknowledging that what we know about is the technology. What we know about is our users, our customers. But we, we should be, you know, very much be careful in terms of trying to know what's best for the world and how to think about massive geopolitical conflicts and things like that. The best thing that we can do is the West. Is to …

AI assessment note: “No. No, they're not.”

Q Did you go through a phase of getting stuff?

A I did. And I was lucky. I went through it. Uh, in my early twenties. And a lot of stuff. And then I got rid of all of it. And, uh, and I think the, for me the sheer realization is that it's the, it's the journey with the people. The outcomes, you know, that we want to see in the world. There's a, they're part of the obsession. They're the, I, if you can see a future that looks like one that ends up being this incredible future, you want to build it. But that moment, once you reach that, which will always be an ever moving goal, is not the interesting moment. The interesting moment is every single day with the people. And I think for me, I just learned more and more over the years is that I, I love people and, and I love the people I work with on my team. I think in every single person, there's something incredible. And, and, If you cut all the stuff, you cut all the money, and, and you pick the hardest, biggest thing you could possibly, you know, focus your life on, and then do it with amazing people, you get to have this incredible experience.

AI assessment note: “I did. And I was lucky. I went through it. Uh, in my early twenties.”

Q Does cash correlate to compute? And what I mean by that is, if you have cash, Can you go to your store and say, I want this amount of compute, please? Or is it more than that?

A It's the right question. I think, again, it depends on how much cash and how much compute. About a year and a half ago when we started as a company, there was a true imbalance between supply and demand in the world that even as a, as you know, frontier AI company starting this, everyone wants you to win. NVIDIA is incentivized to hyperscale. Everyone is incentivized actually to make early stage companies succeed We compute. It's a lot easier when you're an early stage AI company or frontier AI company in general to get compute than it is when you're an enterprise, because they understand this is where the future is heading towards. But even then, there was a real mismatch between demand and supply, and we had to do an incredible amount of work of, of understanding the market, building relationships, and having plan A to Z to get there. In the last six months, I think the world has still a huge supply shortage. And we can, and we can see this, but when you are, where if you're an early stage startup, there's lots of paths for you. If you're a frontier AI company, you need to make decisions about who do you partner with? Who do you work with? How much do you do yourself? You need to, I'm making decisions today that will impact us on compute in 12 or 18 months from now. It's very rare to be at early stage companies where you have to make decisions right now that impact you, you kn…

AI assessment note: “understanding the market, building relationships, and having plan A to Z to get there.”

Q When we look at that talent, when we think about work ethic, it's one thing which Europe is often chastised for. In terms of work-life balance, how do you approach that and feel about implementing standards of work with teams?

A There was a tweet, and if I recall correctly, it's from Aaron, Levi from Box, early on in post-ChatGPT moment, and he wrote something along the lines of, if you feel like you're working extremely hard on reasonable hours as AI is now booming, you're probably right to do so, because it's in these first years that, and I'm probably going beyond what the tweet said, but I think it's in these first years it's where the table gets set. Who has earned the right to be in the race to AGI? And the way that I've, I've always looked at this is from a personal perspective, and so has my co-founder, and so has Margarita, like, has, has put us in a place where we're gonna look back on this moment 10 years from now, just like we would look back to the moment of mobile, internet, you know, first personal computer, and realize that that was the moment where the table got set. And you do not want to, you do not want to look back on that moment and not have given it everything you've got, because it's a race. And look, most startups are not racist. Most startups are against your, yourself. But AGI is a race, and so our view always has been, is that the team that we build is a team that is deeply passionate to be in that race, and, and knows that, and knows And frankly, when you decide to join a race and you're upfront about it, you decide to be, try to become the gold medalist in swimming, that m…

AI assessment note: “there are sacrifices that come with that. You don't get to have it all.”