The Exchanges

Q And you went so far as to suggest This might allow them to pass the best proprietary models coming out of the US, um, yet this year, maybe by Q four of this year. So why don't we dig in there? What is your theory of the case? Why is China, you know, doing so well in open source? And should US model companies like OpenAI and Anthropic be concerned?

A Yeah. So, uh, let, let's kind of tie it into, I think three important things that we see happening. The first one being, you know, the Chinese and, and the president addressed this at the AI summit, right? He addressed, um, the, the point around using copyrighted work and he says, you know, he used a great example. If you read a book and you use it, you're not violating the copyright there. And so he addressed that concern. And that was one of the major things that, you know, a lot, a lot of people didn't talk about, but I think it's important for the model makers. And so the Chinese just have been able to work around that because of, you know, their position on IP. And what we're really seeing here, and I think, you know, Bill teed it up even better off of my tweet, which is, um, you know, they're able to compound. So what you're seeing very quickly is both the open source nature, the open weights nature, uh, allow them to basically compound on each other. So instead of working in silos and instead of having to create giant training clusters, Um, separately. They can basically take each other's work, build on top of it, almost consider it like a remix of someone's model. K-II, sort of a well-known remix of what, what Deep Seek had done, and now we're starting to see that happen really fast, and we're seeing two dimensions of it going quickly. One, we're seeing the leading edge…

AI assessment note: “both the open source nature, the open weights nature, uh, allow them to basically compound”

Q everybody in the world to use that model. And do you believe they have a shot at out-competing, being the upper right, you know, by the end of the year? And if so, do you think that will be the outcome? Like these companies that are using Quen on Grok, do you think they would prefer to use OpenAI so long as it was equally capable and equally priced performance?

A So two things that we see is brand and, you know, the US domiciled or, you know, someone that they can kind of point at that wins. And so if that shows up, it will win because if you're a company and, you know, at some point you have to, you know, get your teams to sign off on what is it that you're using? What are the risks associated with it? And like, you know, who is liable if something goes wrong? And so I sort of feel like, Um, with OpenAI's release and, you know, Meta charges back, or even if some of these startups emerge that, you know, we can point at, I think we'll see a huge shift back towards those models versus, versus the Chinese ones.

AI assessment note: “I think we'll see a huge shift back towards those models versus, versus the Chinese ones.”

Q and he talked about how he just built layer after layer after layer of the stack, you know, over the course of the last decade and a half. But when he said that, Sonny, I know you had a reaction to it, right? Even though you know it's not just a GPU company, When he really broke it down, it seemed like, you know, he did break new territory here.

A Yeah. Like what was great to hear from him and really, you know, positive for, you know, folks thinking about where Nvidia lives in the stack right now is he kind of got into details and then the sub details below CUDA. And he really started going into what they're doing very particularly on mathematical operations to accelerate their partners and how they work really closely with their partners. You know, all the, the cloud service providers. To basically build these functions so that they can further accelerate workloads. The other little nuance that I picked up in there, he didn't focus purely on LLMs. He talked in that particular area about how they're doing that for a lot of traditional models and even newer models are being deployed for AI. And I think just really showed how they are partnering much closer on the software layer than the hardware layer alone.

AI assessment note: “what was great to hear from him and really, you know, positive”

Q mix? And he said, of course. Right. But again, I think conventional wisdom is all around the size of clusters and the size of training. And if, if models don't keep getting bigger than their relevance will dissipate, but he's basically saying every single workload is going to benefit from acceleration, right? It's going to be an inference workload and the number of inference interactions is going to explode higher.

A Yeah. One, one technical detail, which is you need bigger clusters if you're training bigger models. But if you're running bigger models, you don't need bigger clusters. It can be distributed. It can be distributed. Right. And so I think what we're going to see here is that the larger clusters will continue to get deployed. And as Bill said, they'll get deployed for folks, maybe a limited number of folks that need to deploy it for a hundred billion dollar runs or even bigger than that. But you'll see inference clusters be large, but not as large as a training clusters and be a lot more distributed because you don't need it to be all in the same place. And I think that's what'll be really interesting.

AI assessment note: “if you're running bigger models, you don't need bigger clusters. It can be distributed.”

Q And make the argument, why, why are they faster? Why are they cheaper in your mind? But yet, notwithstanding that fact, NVIDIA is going to do, let's call it, 50 or sixty billion of inference this year. Um, and these companies are, you know, still just getting started, right? Why is their inference business? Is it just because of installed base?

A Yeah, I think it's a combination of install base. And I think it's because that inference market is growing so incredibly fast. I think if you're making this decision, even 18 months ago, it would be a really difficult decision to buy any of those three companies because your primary workload was training. And the, you know, the first part of this pod, we talked about how they have such a strong tie in integration to getting training done properly. I think when it comes to inference, you can see all the non NVIDIA folks can get the models up and running right away. There is no Tie into CUDA that's required to go faster. That's required to get the models running, right? Obviously none of the three companies run CUDA. And so that moat doesn't exist around inference.

AI assessment note: “I think it's a combination of install base.”

Q And you went so far as to suggest This might allow them to pass the best proprietary models coming out of the US, um, yet this year, maybe by Q four of this year. So why don't we dig in there? What is your theory of the case? Why is China, you know, doing so well in open source? And should US model companies like OpenAI and Anthropic be concerned?

A Yeah. So, uh, let, let's kind of tie it into, I think three important things that we see happening. The first one being, you know, the Chinese and, and the president addressed this at the AI summit, right? He addressed, um, the, the point around using copyrighted work and he says, you know, he used a great example. If you read a book and you use it, you're not violating the copyright there. And so he addressed that concern. And that was one of the major things that, you know, a lot, a lot of people didn't talk about, but I think it's important for the model makers. And so the Chinese just have been able to work around that because of, you know, their position on IP. And what we're really seeing here, and I think, you know, Bill teed it up even better off of my tweet, which is, um, you know, they're able to compound. So what you're seeing very quickly is both the open source nature, the open weights nature, uh, allow them to basically compound on each other. So instead of working in silos and instead of having to create giant training clusters, Um, separately. They can basically take each other's work, build on top of it, almost consider it like a remix of someone's model. K-II, sort of a well-known remix of what, what Deep Seek had done, and now we're starting to see that happen really fast, and we're seeing two dimensions of it going quickly. One, we're seeing the leading edge…

AI assessment note: “both the open source nature, the open weights nature, allow them to basically compound”

Q with the fact that he said everything in the world today is becoming highly machine learned, right? Almost everything that we do, he said, almost every single application, Word, Excel, PowerPoint, Photoshop, AutoCAD, like it all will run on these modern systems. Sonny, do you buy that? Do you buy that? You know, when people go to replace, you know, compute, they're going to replace it on these modern systems.

A So when I was listening to it, I was buying it, but then when I, he said one thing that kept resonating in my mind, which he said, inference is going to be a billion times larger than training. And if you kind of double click into that, these old systems aren't going to be sufficient enough. Right. If you're going to have that much more demand, that much more workload, which I think we all agree, then how is it that these old systems, which are being decommissioned from training are going to be sufficient? So I think that's where that argument didn't hold, just didn't hold strong enough for me. If that grows as fast as he says, it is as fast as, you know, you guys have seen it in their numbers, then it's going to be a lot more net new inference related, uh, you know, deployments. And there, I don't think. That, that argument holds on the, the transfer from older hardware to newer hardware.

AI assessment note: “I was buying it, but then... that argument didn't hold, just didn't hold strong enough”

Q everybody really holding their breath and hoping that we see something really capable and powerful, uh, is this open source model that's been promised out of OpenAI. Now, of course, Elon says he's committed to open source as well. Grok four is a great model, is impressive what they, uh, what they put out there. Any idea, uh, Sonny, about the open source plans out of, out of, uh, X?

A Yeah, so I think, like, Elon's been, you know, pretty, said it on Twitter that they'll always open source one generation back, and so while they were on three, you know, they should have gotten to two, and now they're on four, so I think the, the thinking is that they will get there. You know, my, my only guess would be is that, um, you know, right now, if they were to open source two or even three, it's so far behind that, um, you know, what's, what's the purpose in doing it? It may not even be utilized, and you'll maybe end up Having to deal with just, you know, a bunch of internet or Twitter FUD. Just coming back to OpenAI, I, I will tell you, it's like one of those things that really you rarely see in the enterprise. It's almost like the demand for like a, you know, like a Tesla Roadster or something, or a Model Y before it came out. Everybody asks for it. It's like, that's the model that everybody wants to use right now. And so, you know, we can't wait till it comes live and, And it, you know, when it's on Grok, when it's all over the world, I think it's going to be a real big one for everyone.

AI assessment note: “Elon's been, you know, pretty, said it on Twitter that they'll always open source”

Q everybody really holding their breath and hoping that we see something really capable and powerful, uh, is this open source model that's been promised out of OpenAI. Now, of course, Elon says he's committed to open source as well. Grok four is a great model, is impressive what they, uh, what they put out there. Any idea, uh, Sonny, about the open source plans out of, out of, uh, X?

A Yeah, so I think, like, Elon's been, you know, pretty, said it on Twitter that they'll always open source one generation back, and so while they were on three, you know, they should have gotten to two, and now they're on four, so I think the, the thinking is that they will get there. You know, my, my only guess would be is that, um, you know, right now, if they were to open source two or even three, it's so far behind that, um, you know, what's, what's the purpose in doing it? It may not even be utilized, and you'll maybe end up Having to deal with just, you know, a bunch of internet or Twitter FUD. Just coming back to OpenAI, I, I will tell you, it's like one of those things that really you rarely see in the enterprise. It's almost like the demand for like a, you know, like a Tesla Roadster or something, or a Model Y before it came out. Everybody asks for it. It's like, that's the model that everybody wants to use right now. And so, you know, we can't wait till it comes live and, And it, you know, when it's on Grok, when it's all over the world, I think it's going to be a real big one for everyone.

AI assessment note: “said it on Twitter that they'll always open source one generation back”

Q uses the tool and it searches the internet. Um, so if you don't have to compress all this Wikipedia information, I don't know, take it, take a subject like World War II, you just need to know how to go out and use the internet to find the information, to summarize it in real time. How has that changed the pace of progress and the balance between open and closed?

A Yeah, and so it's spot on, right? Brad, what we see now, and you see it when you use these reasoning models, and what I suggest everyone do is when you're using a reasoning model, you can usually expand out its thought process. So when you ask a question, it'll say, oh, the person is asking a question about this. What should I do? Let me go and maybe search the internet. Let me go do a few different things that can have a lot of different tools. And so the push has been towards, you know, really, really strong reasoning models. And, you know, we have to give credit there. OpenAI really started that. With oh, one, that was really the first reasoning model that was put out there. But I think on the back of the research and the back of it, you know, everyone talked about and, you know, the pod's good friend, Noam Brown was the leader on that program. Everyone's been able to look at that and say, let's reframe the problem. And this allows us to build stronger reasoning models that don't have to compress, like you said, all the internet's information. And once they're coupled with strong tools, you start getting these really, really incredible results that don't even just show up in benchmarks because none of the benchmarks Really allow you to use a tool to answer the results. And if they did, we're going to see a whole bunch of new set of results that happen there.

AI assessment note: “allows us to build stronger reasoning models that don't have to compress”

Q uses the tool and it searches the internet. Um, so if you don't have to compress all this Wikipedia information, I don't know, take it, take a subject like World War II, you just need to know how to go out and use the internet to find the information, to summarize it in real time. How has that changed the pace of progress and the balance between open and closed?

A Yeah, and so it's spot on, right? Brad, what we see now, and you see it when you use these reasoning models, and what I suggest everyone do is when you're using a reasoning model, you can usually expand out its thought process. So when you ask a question, it'll say, oh, the person is asking a question about this. What should I do? Let me go and maybe search the internet. Let me go do a few different things that can have a lot of different tools. And so the push has been towards, you know, really, really strong reasoning models. And, you know, we have to give credit there. OpenAI really started that. With oh, one, that was really the first reasoning model that was put out there. But I think on the back of the research and the back of it, you know, everyone talked about and, you know, the pod's good friend, Noam Brown was the leader on that program. Everyone's been able to look at that and say, let's reframe the problem. And this allows us to build stronger reasoning models that don't have to compress, like you said, all the internet's information. And once they're coupled with strong tools, you start getting these really, really incredible results that don't even just show up in benchmarks because none of the benchmarks Really allow you to use a tool to answer the results. And if they did, we're going to see a whole bunch of new set of results that happen there.

AI assessment note: “this allows us to build stronger reasoning models that don't have to compress”