The Exchanges

Q Given where we are in the stack, a lot of the complexity that we have, we, we have a margin structure that's a little bit different to like traditional SaaS being 80% I don't know the margins precisely here, but they traditionally said in the 30 to 40% range for where we are. Is that the new normal for where we are?

A I don't think that's new normal. I think that is a reflection, at least for us, I don't know other companies. Uh, for us, it is a reflection of we are in a hyper growth phase. Um, during hyper growth phase, you have the choice, right? You either optimize, to me, margin optimization is a constraint problem, as in, hey, we want to go to 70% margin, we want to go to 80% margin, And, and then we are going to go backwards and impose those constraints to guarantee those margin, and usually constraints slow down, um, innovation. So, for, give you an example, uh, during system development, and, uh, in a high velocity, uh, system expanding phase, we, we don't want to overbuild, because we're in kind of High experimentation while testing, uh, you know, what will stay, what will not stay, optimization doesn't make any sense. Once we know this is a system that we want to build a hundred percent, then, and we are going to scale this a thousand times bigger, then we go optimize the heck out of it. I think, might not, you will think about business the same way. We're in the hypergrowth, um, if our focus is only Optimize growth margin. We absolutely can do that, but we are sacrificing the speed of growth as well because we want to go everywhere. We want to go into different, ah, geo-regions. Ah, we want to go into tackle different use cases. We want to create, constantly create, um, different …

AI assessment note: “I don't think that's new normal. I think that is a reflection”

Q What did you wait on in the fireworks journey that you wish you hadn't waited on?

A Marketing. We talk about it. So we are a little bit nerdy in this way, that at the very beginning of our journey, we kind of, we didn't discuss it, but we feel product will speak for itself. At the end, product stands, and we want to devote all our effort and focus on building product, working with the customer, um, validate product market fit, and, and go from there. And we didn't spend much time marketing at all. We didn't prioritize educating our customer what's the right direction to think about the trend, um, and, ah, and the value. Um, but we do think, now I do think it's important. Marketing is not about flaws. It's more about education. Um, it's more about clarity. Um, and, uh, and we are working on that.

AI assessment note: “Marketing. We talk about it.”

Q Pretty much only hire immigrants. British people don't work very hard. Uh, sorry. Uh, very scientific and rigorous. Use data for most things. I actually think creativity often comes from data. And is informed by data. Um, and unwaveringly like accountable and ownership. Like nothing is anyone else's fault. It's all my fault. Even if it's someone else's fault, that's a 20 VC person. What would you say yours is?

A It's not in weird way. It's not competence. It's weird. We need, we want people with the high confidence, uh, competence. Um, but more importantly, the strong indicator where they will do well, Um, in this wave, especially in fireworks, is whether, um, they are really built for taking stream ownership. Um, extreme ownership as in we are not putting people anywhere in any boxes, and we're just stacking the box together into a tower. Uh, we, people just automatically claim, hey, this is an end-to-end problem. I'm gonna see through the whole thing and work with a bunch of people to make it happen, and, uh, and I'm gonna deliver it no matter what. So those kind of people has the highest, longest mileage. And their growth curve is amazing also.

AI assessment note: “they are really built for taking stream ownership”

Q Okay, what have you changed your mind on most in the last 12 months?

A I think how fast we grow, I changed my mind, because I have been quite worried about too big a team too early. Uh, so that's why when I met George, I told him, we're too small for you, because I don't intend to grow very fast in terms of people. Uh, I worry about Slow down, getting slowed down, and lose agility and velocity very deeply. So, um, but since then, it's, we have been very aggressively using AIR tools. We have developed our own unique way of hiring certain type of people that we know, ah, they will be charging forward with high velocity as extreme sense of ownership, ah, very communicative. And never take no answer. So we also learn how to, how to get those people. Uh, and now I feel much more comfortable skating really fast.

AI assessment note: “I think how fast we grow, I changed my mind”

Q I'm sorry to be harping on that. Why does Jensen skip your layer of the cake? Because he's doing Nemo Tron with models. Why does he not want to cannibalize your business too?

A Well, Jensen is not building a cloud either, right? You can say, hey, Jensen probably have all the rights to build an Nvidia cloud. Uh, so he's not building a cloud infrastructure. Um, I think he mentioned that as well. I mean, people asked him that question. Um, and he also mentioned he want to specialize in what they have the rights to do. Uh, why models? I think it's pure, um, a supply chain question is If U.S. doesn't have a U.S. native open model, it's a problem. It's a supply chain problem. So, so he is solely there to solve the supply chain problem, but if there's no supply chain problem because the company like us are providing this specialized intelligence platform layer, then he doesn't need to worry about it. So he just want to make sure the whole entire five layers of AI cake is flowing. There's no blockage. And if there's a blockage, you know, he's interested in solving those problems.

AI assessment note: “if there's no supply chain problem because the company like us are providing”

Q Given where we are in the stack, a lot of the complexity that we have, we, we have a margin structure that's a little bit different to like traditional SaaS being 80% I don't know the margins precisely here, but they traditionally said in the 30 to 40% range for where we are. Is that the new normal for where we are?

A I don't think that's new normal. I think that is a reflection, at least for us, I don't know other companies. Uh, for us, it is a reflection of we are in a hyper growth phase. Um, during hyper growth phase, you have the choice, right? You either optimize, to me, margin optimization is a constraint problem, as in, hey, we want to go to 70% margin, we want to go to 80% margin, And, and then we are going to go backwards and impose those constraints to guarantee those margin, and usually constraints slow down, um, innovation. So, for, give you an example, uh, during system development, and, uh, in a high velocity, uh, system expanding phase, we, we don't want to overbuild, because we're in kind of High experimentation while testing, uh, you know, what will stay, what will not stay, optimization doesn't make any sense. Once we know this is a system that we want to build a hundred percent, then, and we are going to scale this a thousand times bigger, then we go optimize the heck out of it. I think, might not, you will think about business the same way. We're in the hypergrowth, um, if our focus is only Optimize growth margin. We absolutely can do that, but we are sacrificing the speed of growth as well because we want to go everywhere. We want to go into different, ah, geo-regions. Ah, we want to go into tackle different use cases. We want to create, constantly create, um, different …

AI assessment note: “I don't think that's new normal. I think that is a reflection”

Q I thought so too, ok? Again, I, I, I admit to being a moron, which is why I think the show is a little bit successful. I thought so too, but then how come everyone is seemingly doing it as if it's like just another product? As I said, OpenAI, Anthropic, DeepSeek, Meta, we're building our own chips now.

A I think MATA has been building their chips for more than five years, way more than five years. Um, and MTIA has been a project since, you know, 2018, um, maybe earlier. So, because MATA has been investing in AI for a long time, pre-gen AI. Um, and they have a huge, uh, AI workload to focus on ranking recommendation. And MATA has been building other Hardware as well in the past. So whenever the, the, the usage has passed certain threshold, it make economic sense for you to build it, build the underlying supply. Right? So, uh, and then you can specialize towards your workload. And, uh, that's another form of specialization is specialize to bake your logic into hardware and this hardware is purpose-built for your particular workload. And you better, uh, You better make sure this workload doesn't change. Because it's really hard, once the hardware is taped out, it's really hard to go back and change it. It's very, it's possible it's very costly. Um, so once your workload stabilizes, once your business stabilizes, it doesn't change too often, then that's the time, um, to consider building a chip. I still see the whole AI world, especially models, customization, It's very dynamic. Very, very dynamic. Workload pattern is very dynamic. So think about how much energy in the application space. People are experimenting all kind of things. You don't know which one is going to take off, and…

AI assessment note: “whenever the, the, the usage has passed certain threshold, it make economic sense”

Q What's your biggest lesson from working with Jensen Huang on what makes him so special?

A He's everywhere. I seriously think he has a clone of, like, hundreds of Jensen somehow. Um, for example, I sent him an email. He will reply in one minute. I, I just don't understand how he's, like, constantly, um, in details, and, but, but now I operate a company for four years, I understand why he's doing that, is that's, that defines velocity, because what is leadership? Leadership is just judgment. It's not privilege. It's judgment. It's, you basically have the, Context. You need to have the right context to make the right judgment for the team. If, and especially in a high velocity space, if you do not know what's happening, what works, what doesn't work, what are the gaps, you make the wrong call. Um, in a slow moving space, you, you, you, you can wait for the cascading information up and down and make those calls, but in a fast iteration space, You just cannot wait. Um, because it's guaranteed, there is information loss in, in, in transition. Layer after layer. People after people. It always happens. And not knowing what exactly is happening and make, having the precision of make judgment makes bad leadership. And he is demonstrated through his own example Even before this crazy AI thing, he's operating that way. And before I was admiring him in his sheer amount of volume of capability of doing that, now I understand the wisdom behind that because I also operate that way.…

AI assessment note: “I understand why he's doing that, is that's, that defines velocity”

Q What did you wait on in the fireworks journey that you wish you hadn't waited on?

A Marketing. We talk about it. So we are a little bit nerdy in this way. At the very beginning of our journey, we kind of, we didn't discuss it, but we feel product will speak for itself. At the end, product stands, and we want to devote all our effort and focus on building product, work with the customer, validate product market fit, and go from there. And we didn't spend much time marketing at all. We didn't prioritize educating our customer what's the right direction to think about the trend and the value. But we do think now, I do think it's important. Marketing is not about flaws. It's more about education. It's more about clarity. And we are working on that.

AI assessment note: “Marketing. We talk about it.”

Q That's quite late. Can I ask, how do you reflect on being a 48 year old founder when we glorify starting a company when you're pretty much 15 these days?

A Um, I, I didn't think deeply about that. I always want to have a tech business myself. I actually want to start a business in 2015. Uh, because I, I'm a first generation immigrant, I came to US in 2000. I did my PhD in distributed system, uh, computer science, especially focused on databases. And database is a very concept system to build, a lot to, a lot of different objectives to optimize for, and pretty much touched, after I, uh, joined research lab, I pretty much touched every single aspect of processing data. And then I moved to LinkedIn to kind of further it down to build, Systems and products, um, to be used to drive real impact. At that time, I feel I'm ready to start a company. I know all the tech. I know what product to build. I have a business proposal. I have a list of people I want to start a company with. And I spent time thinking about it, and I paused. Because I don't think I have the skill set on people to build a company. It's not just about product. It's not just about tech. It's actually about people. And I decided I want to go to a place I can learn the most of, uh, of people. And the best company at the time is Facebook. Uh, it's a rising star, uh, in Silicon Valley. And secretly, I was planning to learn for one year or two, and even go back to do my own business. I stayed there for seven years.

AI assessment note: “I don't think I have the skill set on people to build a company.”

Q That's quite late. How do you reflect on being a 48 year old founder when we glorify starting a company when you're pretty much 15 these days?

A I didn't think deeply about that. I always want to have a tech business myself. I actually want to start a business in 2015, because I, I'm a first generation immigrant. I came to US in 2000. I did my PhD in distributed system, computer science, especially focused on databases. And database is a very concept, system to build, a lot of different objectives to optimize for, and pretty much touched, after I joined research lab, I pretty much touched every single aspect of processing data. And then I moved to LinkedIn to kind of further it down to build systems and products to be used to drive real impact. At that time, I feel I'm ready to start a company. I know all the tech. I know what product to build. I have a business proposal. I have a list of people I want to start a company with. And I spent time thinking about it and I paused because I don't think I have the skill set on people to build a company. It's not just about product. It's not just about tech. It's actually about people. And I decided I want to go to a place I can learn the most of people, and the best company at that time is Facebook. It's a rising star in Silicon Valley, and secretly I was planning to learn for one year and even go back to do my own business. I stayed there for seven years.

AI assessment note: “I spent time thinking about it and I paused because I don't think I have”

Q I thought so too, ok? Again, I, I, I admit to being a moron, which is why I think the show is a little bit successful. I thought so too, but then how come everyone is seemingly doing it as if it's like just another product? As I said, OpenAI, Anthropic, DeepSeek, Meta, we're building our own chips now.

A I think MATA has been building their chips for more than five years, way more than five years. Um, and MTIA has been a project since, you know, 2018, um, maybe earlier. So, because MATA has been investing in AI for a long time, pre-gen AI. Um, and they have a huge, uh, AI workload to focus on ranking recommendation. And MATA has been building other Hardware as well in the past. So whenever the, the, the usage has passed certain threshold, it make economic sense for you to build it, build the underlying supply. Right? So, uh, and then you can specialize towards your workload. And, uh, that's another form of specialization is specialize to bake your logic into hardware and this hardware is purpose-built for your particular workload. And you better, uh, You better make sure this workload doesn't change. Because it's really hard, once the hardware is taped out, it's really hard to go back and change it. It's very, it's possible it's very costly. Um, so once your workload stabilizes, once your business stabilizes, it doesn't change too often, then that's the time, um, to consider building a chip. I still see the whole AI world, especially models, customization, It's very dynamic. Very, very dynamic. Workload pattern is very dynamic. So think about how much energy in the application space. People are experimenting all kind of things. You don't know which one is going to take off, and…

AI assessment note: “once your workload stabilizes, once your business stabilizes... that's the time, um, to consider building a chip.”

Q How much will token cost come down? Is this, just tell me, is it like a halving? Is it like a, oh, it'll be a 100th of the cost?

A So there, there's different ways to think about this. It's not all tokens are equal. I think we should establish a, um, best practice to evaluate the token economy per task. Because different model are, have different way of spit out tokens. Some are much more verbose than the other. So, uh, so you can imagine one model Um, is two x cheaper than the other, but it's two x more verbose to solve the same task, and then they're the same cost, right? Uh, so, but overall, I think as the model quality improve, I think being precise is going to be part of the, uh, optimization, and so that's one level of optimization is to solve one task, we should, we should need less tokens, okay? Uh, and the second is For one token, um, and how to do that is you, you need to customize the model to solve your problem especially better and more precise. That goes into model tuning. Um, and second is for each token spit out from those models and processed by those models, we also specialize in making the unit economists much better, uh, through our platform. And third is underlying infrastructure, uh, like the GPUs, the surrounding, like memories, and all this. Today is under stark supply chain constraint is gonna get much better. Situation will get much better. I don't think, probably in the next one year or a year or a half, this situation will not change. But in the long term, two to three years, it…

AI assessment note: “overall, I can imagine, you know, 10 X, uh, cost reduction in the next three years”

Q What area of AI is under-invested in today in your mind? You mentioned, like, cooling or servers. What area is, like, under-invested in?

A I think AI has the Sexy part of this is such an innovative, creative technology, and the build something on top of it is the focus, but monitoring the ROI, I think the industry start to kind of pay attention to it, but eventually that's what matters, um, is not how much spend is, how much, what is the return, and, and, and what is the cost, and what is the attribution, So I, I think in the next couple of years as AI is getting more and more into production, there will be a lot of focus in getting that clarity and getting that discipline out. So, uh, the token maxing is just, um, I think, a thing in time, um, but we'll quickly move into ROI maxing, which is about all about running a business.

AI assessment note: “monitoring the ROI... is not how much spend is, how much, what is the return”

Q When 90% of enterprise workflows can be done, as you said, that the incredible array of functions that you now use open source for with open models, so the usage for frontier models will not be as large as it was if it was needed for everything. So are these companies actually dramatically overvalued and overestimated if the majority can just go through open?

A I think people start to realize it. I remember from two years ago, uh, I went to different places and talked about an interesting phenomenon. Um, that doesn't exist in the past in the SaaS era. During SaaS time, product market fit and the durable business almost are equivalent to each other. The hardest thing is find power market fit, and then once you find it, it just scale as fast as you can, right? Because CPU is a commodity. The infrastructure you build on top is almost like a commodity. You don't even worry about that as your cogs. And now, product market fit and durable business are two separate concepts. Um, for startups, you know, we have great companies that have product market fit. Customers want to pay them, and they really value their product, but they cannot scale, because once they scale, they could scale into bankruptcy. Have you heard about scaling to bankruptcy? So that's a real problem. Um, it's even a bigger problem for incumbents. So the big companies for digital native, um, because they have the traffic. They have a huge amount of traffic. They're the winner from a decade ago when they were startups. And they have so much traffic. Once they draw out those AI features, they're going to reach to all their customer base and they cannot afford to do it. Because the CFO look at their, um, cost proposal. Cost forecasting is kind of, there's no way you can justify…

AI assessment note: “I think people start to realize it.”

Q What do you see as the greatest bottleneck today? You know, I think it was when I had Jonathan from Grok on the show who said, like, HBM was the greatest bottleneck, and that's why you've seen the five X increase in price. What do you see as the greatest bottleneck that people don't talk about enough?

A I still think we don't have a great system for very large model. I really believe the fundamental lower level infrastructure costs will go down. For solving tasks, we should need less token. That will increase. So collectively, um, the cost was significantly reduced. Uh, therefore, We can run the highest intelligence model much more ubiquitously in the future, but we don't have a system designing for that. For example, we don't have a great system designed for, um, 10 trillion parameter models today, and that will require very smart engineer co-design from the model to, um, the customization serving platform layer all the way to chip layer. The chip is not the individual chip, but the system, collection of chips in system, um, and all as a total package. I think there's still a lot of innovation we can do.

AI assessment note: “I still think we don't have a great system for very large model.”

Q Pretty much only hire immigrants. British people don't work very hard. Uh, sorry. Uh, very scientific and rigorous. Use data for most things. I actually think creativity often comes from data. And is informed by data. Um, and unwaveringly like accountable and ownership. Like nothing is anyone else's fault. It's all my fault. Even if it's someone else's fault, that's a 20 VC person. What would you say yours is?

A It's not in weird way. It's not competence. It's weird. We need, we want people with the high confidence, uh, competence. Um, but more importantly, the strong indicator where they will do well, Um, in this wave, especially in fireworks, is whether, um, they are really built for taking stream ownership. Um, extreme ownership as in we are not putting people anywhere in any boxes, and we're just stacking the box together into a tower. Uh, we, people just automatically claim, hey, this is an end-to-end problem. I'm gonna see through the whole thing and work with a bunch of people to make it happen, and, uh, and I'm gonna deliver it no matter what. So those kind of people has the highest, longest mileage. And their growth curve is amazing also.

AI assessment note: “whether, um, they are really built for taking stream ownership. Um, extreme ownership”

Q Do you have to be full, we're going to Jensen's five-layered AI cake, do you have to then be full stack to win or to reduce dependencies? We've seen OpenAI come out with Jalapeno, terrible name, Anthropic talking to Samsung about building their own chips, DeepSeeker building their own chips, Zuck came out with Meta building their own chips. Do you have to be all of it?

A It really depends on the company philosophy. To us, agility is everything, and we need to earn the rights of building anything. So focus is everything for us, and we want to focus on where we add the biggest amount of value based on our strength. We would like to leverage other people's strength to build on top of. So in particular, we want to run everywhere, all possible air chips in the world. We don't want to limit it by how much chips we can bring into our data center, whether we're constructed or rented. But over time, when the business grows very big, right, so I still remember when Matter was young, they, they don't build everything. And when they're big, they make sense to build. You earn the rise to, to build for your own, you know, giant traffic, and this, if it save, like, five times more cost, then you should go do it, right? So, but I think at the early stage, that's why I give, tell you an interesting story in the coding space. I would say Curso is the first company they have decided to work with us. Early on. I still remember when they worked with us, they were single digit million dollar. Very small. That's only two years ago. They grew by a hundred, a thousand X over two years. But they decided to work with us early on because they recognize they only want to focus on product innovation and later on research. They do not want to focus on, you know, this platfor…

AI assessment note: “We do not want to kind of own the whole entire stack.”

Q That's quite late. Can I ask, how do you reflect on being a 48 year old founder when we glorify starting a company when you're pretty much 15 these days?

A Um, I, I didn't think deeply about that. I always want to have a tech business myself. I actually want to start a business in 2015. Uh, because I, I'm a first generation immigrant, I came to US in 2000. I did my PhD in distributed system, uh, computer science, especially focused on databases. And database is a very concept system to build, a lot to, a lot of different objectives to optimize for, and pretty much touched, after I, uh, joined research lab, I pretty much touched every single aspect of processing data. And then I moved to LinkedIn to kind of further it down to build, Systems and products, um, to be used to drive real impact. At that time, I feel I'm ready to start a company. I know all the tech. I know what product to build. I have a business proposal. I have a list of people I want to start a company with. And I spent time thinking about it, and I paused. Because I don't think I have the skill set on people to build a company. It's not just about product. It's not just about tech. It's actually about people. And I decided I want to go to a place I can learn the most of, uh, of people. And the best company at the time is Facebook. Uh, it's a rising star, uh, in Silicon Valley. And secretly, I was planning to learn for one year or two, and even go back to do my own business. I stayed there for seven years.

AI assessment note: “I didn't think deeply about that. I always want to have a tech business myself.”

Q Final one for you. What does no one see about the next three years that you see very clearly happening or not happening?

A I really see people will own their, every single company will own their own intelligence as a must-have. It's not optional. That's a trend I'm seeing, um, because there's an analogy to software is there's a reason why every company build their own software stack. There's no standardized software you just use off the shelf to solve your problem because every single company is solving a unique problem. And they want to be a software because they want to have full control. Um, and obviously they will pick and choose which part of the stack they want to build themselves, which part of the stack is common knowledge, there's no point of building. But every single company owns their own software stack. Obviously we're talking about, um, in the SAS time, right? So, uh, same. I think at that time every single company should own their own intelligence.

AI assessment note: “every single company will own their own intelligence as a must-have”

Q I'm sorry to be harping on that. Why does Jensen skip your layer of the cake? Because he's doing Nemotron with models. Why does he not want to cannibalize your business too?

A Jensen is not building a cloud either, right? You can say, hey, Jensen probably have all the rights to build an Nvidia cloud. So he's not building a cloud infrastructure. I think he mentioned that as well. I mean, people asked him that question. And he also mentioned he want to specialize in what they have the rights to do. Why models? I think it's pure a supply chain question is if U.S. doesn't have a U.S. Native open model. It's a problem. It's a supply chain problem. So, so he is solely there to solve the supply chain problem. But if there's no supply chain problem, because the company like us are providing this specialized intelligence platform layer, then he doesn't need to worry about it. So he just wants to make sure the whole entire five layers of AI cake is flowing. There's no blockage. And if there's a blockage, you know, he's interested in solving those problems.

AI assessment note: “he just wants to make sure the whole entire five layers of AI cake is flowing”

Q Something in inference that the world was not focused on. The world was focused on training. I think it's helpful for people to understand kind of the stack, because beneath you there's obviously kind of chip providers, and you're Nvidia's of the world, and then you've got above you the model providers, and you sit in between. Why is that a valuable part of the stack and not a commodity?

A That's a really good question, but why bother specialized intelligence? Why not just use generalized intelligence and, and you worry less things, right? You just kind of build on top of a, uh, API, uh, that provided by Frontier Labs. Wouldn't bet, wouldn't that be much easier? So the argument is the following. If you think intelligence is a derivative of data, then majority of the data is actually not used for training a general intelligence model. The training data is coming from public internet and the labeled data. Public internet is very small. Corpus of data compared with worse data Majority of words data, actually private data, locked inside application, locked inside enterprise. It will never get shared with anyone else because this is company's proprietary IP. So, so then it's interesting. If you look at the space, then it becomes very interesting because majority of data is not being activated to derive any intelligence, and that's where we believe in is, is to activate that data, and we believe the Future of the frontier of the intelligence are actually private intelligence, are specialized intelligence. So that's kind of where Fire was, from the beginning, we have been focusing on driving the value.

AI assessment note: “Future of the frontier of the intelligence are actually private intelligence, are specialized intelligence.”

Q Something in inference that the world was not focused on. The world was focused on training. I think it's helpful for people to understand kind of the stack, because beneath you there's obviously kind of chip providers, and you're Nvidia's of the world, and then you've got above you the model providers, and you sit in between. Why is that a valuable part of the stack and not a commodity?

A That's a really good question, but why bother specialized intelligence? Why not just use generalized intelligence and, and you worry less things, right? You just kind of build on top of a, uh, API, uh, that provided by Frontier Labs. Wouldn't bet, wouldn't that be much easier? So the argument is the following. If you think intelligence is a derivative of data, then majority of the data is actually not used for training a general intelligence model. The training data is coming from public internet and the labeled data. Public internet is very small. Corpus of data compared with worse data Majority of words data, actually private data, locked inside application, locked inside enterprise. It will never get shared with anyone else because this is company's proprietary IP. So, so then it's interesting. If you look at the space, then it becomes very interesting because majority of data is not being activated to derive any intelligence, and that's where we believe in is, is to activate that data, and we believe the Future of the frontier of the intelligence are actually private intelligence, are specialized intelligence. So that's kind of where Fire was, from the beginning, we have been focusing on driving the value.

AI assessment note: “we believe the Future of the frontier of the intelligence are actually private intelligence”

Q cases by the administration briefly for 19 days. Um, especially in Europe, we suddenly went, oh my gosh, we cannot be at the hands of open AI and anthropic where we can just be banned in our health services, sit on the infrastructure of something that, you know, an administration can turn off. Do we see a future of sovereign models where large nations or nation blocks own sovereign models?

A I definitely see that possibility. Um, I also see, if we think about the general intelligence model as the electricity layer, as a power line, Every country should, should own their own power line, right? So, um, I, I do, I, I think that is a very scary moment, is my power line is going to be cut off, and all my fundamental, um, day to day is going to not working, because I feel so frustrated whenever there's a power outage in my home alone. I feel so frustrated when I cannot access my Wi-Fi. I feel so anxious. So, uh, I mean, Obviously, the, you know, operating the country is, is extremely important built on top of this fundamental, uh, baseline. So, um, and for every single company, same thing. It's not just, you know, whether a country should have their unique, uh, sovereign independence, but every single company should have their independence. Um, you don't want any single person to cut you off. Uh, that's extremely scary moment.

AI assessment note: “I definitely see that possibility.”

Q How do you think about that question for yourself when, when, when you're sitting there in an armchair on a Sunday afternoon thinking, hmm, we're optimizing for growth now. When is that time to optimize for gross margin?

A Well, I would say we optimize, we want to optimize for both. So, so here's how I think about it. Um, optimize for growth is a, requires a lot of business planning. Assuming there's product market fit. Optimize for growth margin is optimized for differentiation. Um, I, I think I want to avoid over-optimizing for growth margin, but we should optimize for growth margin continuously, as in we should optimize for product differentiation continuously. There's no question about it. And, uh, um, I think we want to continue to optimize towards a healthy growth margin, which allow us to grow really fast, and it's a trade-off And we don't want to take compromises. Um, the compromise as in we over optimize growth margin to result in a very slow growth, right? And one possible way to optimize growth margin, we do not grow at all. We just optimize the heck out of it. I know we can heal climb to a high number, but that's absolutely a disaster outcome.

AI assessment note: “Well, I would say we optimize, we want to optimize for both.”

Q How much will token cost come down? Is this, just tell me, is it like a halving? Is it like a, oh, it'll be a 100th of the cost?

A So there, there's different ways to think about this. It's not all tokens are equal. I think we should establish a, um, best practice to evaluate the token economy per task. Because different model are, have different way of spit out tokens. Some are much more verbose than the other. So, uh, so you can imagine one model Um, is two x cheaper than the other, but it's two x more verbose to solve the same task, and then they're the same cost, right? Uh, so, but overall, I think as the model quality improve, I think being precise is going to be part of the, uh, optimization, and so that's one level of optimization is to solve one task, we should, we should need less tokens, okay? Uh, and the second is For one token, um, and how to do that is you, you need to customize the model to solve your problem especially better and more precise. That goes into model tuning. Um, and second is for each token spit out from those models and processed by those models, we also specialize in making the unit economists much better, uh, through our platform. And third is underlying infrastructure, uh, like the GPUs, the surrounding, like memories, and all this. Today is under stark supply chain constraint is gonna get much better. Situation will get much better. I don't think, probably in the next one year or a year or a half, this situation will not change. But in the long term, two to three years, it…

AI assessment note: “overall, I can imagine, you know, 10 X, uh, cost reduction in the next three years”

Q efficiency, you do charge more. You know, when I, when I did the research, when it compared to competitors, I got like, together's price king, and I didn't mean this disparagingly, but like, they're cheaper. If you want cheap, you go there, and respectfully, if you want better quality product, you go to you. But it is more expensive. Do you think that's a fair assessment and a fair analogy?

A I think we're probably not comparing apples to apples in the sense that, uh, again goes back to our business. Majority of our traffic is, um, is customized model. Um, and, uh, we optimize for quality. Number one, always quality. Quality as in model quality, Uh, towards your applications, your specific business, your use case, and so on. The second is, um, when we deliver those models in inference, it's also quality. Um, and we care quality so much, we do extreme things. Um, for example, during training time, there's a very hard thing to achieve. It's called zero KLD. It's a little bit technical. The idea here is- Zero KLD. KLD. KLD is a measure of, uh, of quality. Um, and, uh, what it means is between the training system and the inference system, when model moves over, uh, we have bit equivalence. Uh, so as in the numerics are fully the same. We do not lose a bit of accuracy. Uh, that's really hard to achieve, but the reason we push that We deliver that, um, and the reason we push that is because we know, um, our primary business is in model customization and inference of customized model. Um, and we want our customers, every single dollar, invest in training, maximize it. Um, and then they, if cross-training inference boundary is not bid-wise equivalent, they just drop the quality down. And, and then it's like you pay, you pay your trading investment by, um, discounted quality…

AI assessment note: “I think we're probably not comparing apples to apples in the sense that”

Q I'm sorry to be harping on that. Why does Jensen skip your layer of the cake? Because he's doing Nemo Tron with models. Why does he not want to cannibalize your business too?

A Well, Jensen is not building a cloud either, right? You can say, hey, Jensen probably have all the rights to build an Nvidia cloud. Uh, so he's not building a cloud infrastructure. Um, I think he mentioned that as well. I mean, people asked him that question. Um, and he also mentioned he want to specialize in what they have the rights to do. Uh, why models? I think it's pure, um, a supply chain question is If U.S. doesn't have a U.S. native open model, it's a problem. It's a supply chain problem. So, so he is solely there to solve the supply chain problem, but if there's no supply chain problem because the company like us are providing this specialized intelligence platform layer, then he doesn't need to worry about it. So he just want to make sure the whole entire five layers of AI cake is flowing. There's no blockage. And if there's a blockage, you know, he's interested in solving those problems.

AI assessment note: “he just want to make sure the whole entire five layers of AI cake is flowing”

Q You said cursor being the front runners in terms of how innovative they've been. I completely agree with you, but I heard, and you know, I, I really stalk you before shows, but I heard that, you know, CTO Dima was embedded at cursor for months building the RL infrastructure. Is that how it has to be done? And is that scalable?

A So what's happening is Usually in the early adoption curve of new technology, the early adopters are all hackers. A hacker is not in a bad way. It's not, it doesn't have a negative connotation. They, they have deep expertise in certain area, and they want to control a lot of things. Um, versus in the late stage of a new tech adoption curve, it starts to get more accessible, um, to a much bigger cohort user. Doesn't have deep Expertise, and they, they need less control. So it always go into deep control first, usually, and, ah, little control later. So we definitely are aiming towards the later stage as the ultimate time we want to target, but it's also extremely valuable to understand, ah, what is required to get there. So, so that's why we partnered deeply with Cursor. They are the pioneer trying those ideas. They do have researchers from frontier labs, and they want to control every single thing, and at the same time, we're also pushing to the boundary. We're doing, we're doing things that never existed before. We're doing things that never existed before, because we push the boundary that is unique, ah, to, to this particular setting. Ok, what is unique is here. Typically, if you think about training, training happens, training is very capital intense. Um, and, uh, and it usually happens in big companies. They have a lot of money. They put those money to buy very expensive t…

AI assessment note: “aiming towards the later stage as the ultimate time... but it's also extremely valuable”

Q Can I, can I ask you a question bluntly, which is incredible customers have amazing progress they've had with you, um, and it's wonderful to see that partnership. It's a very large customer for you. How do you think about the concern of a cursed churn in the wake of a SpaceX acquisition?

A Yeah. Everyone's concerned. The whole entire industry in terms of In terms of application innovations by model, in the sense there are few companies that are very, very successful. They escape velocity, but few of them. So that's the shape of the whole entire industry. And last year, Cursor is one of the few. Um, I would say all model companies are concentrated on Cursor. We concentrate on the same group of, um, app companies. And, um, and since then it, it does change, right? So we do have a very healthy, diversified customer base. Um, especially, I think last year is the year of coding. Uh, I think all major coding companies are on us. And this year is the year of co-work. And co-work is much more diversified by itself. Then coding. Because there's general purpose co-work, for example, general purpose, like, co-work to help you do all kind of research. Um, you want to ask, hey, what will be the, um, what will be the NVIDIA GPU price, uh, two years later? Um, what will be Anthropix stock price after IPO? So those are deep research. General purpose deep research. Um, Or there are so many different categories of special purpose co-work. Legal, we just talked about two great legal companies. Finance, customer support, recruiting, sales, marketing, uh, healthcare. So there's very broad set of co-work space of innovation app company. They are doing really well, and we have them as …

AI assessment note: “So we do have a very healthy, diversified customer base.”