The Exchanges

Q at the time, I, I think most people, even people in tech were like, What is this? Were you involved in it at all? Like, you know, because you were so connected to the researchers, to Ilya, taking that talent out of Google and Facebook, to be blunt, but reseeding the research community and opening it up, um, was such an important moment. Were you involved in it at all?

A I wasn't involved in the founding of it, but I knew a lot of the people there, and, um, uh, Elon, of course, uh, I knew, and, uh, uh, Peter Beal was there, and Ilya was there, and, Uh, we have, we have some great employees today that were there in the beginning, and I knew that they needed this amazing computer that we were building, and we're building the first version of the DGX, which, you know, today when you see a hopper, it's 70 pounds, 35,000 parts, 10,000 amps, but DGX, the first version that we built was, uh, used internally, and I delivered the first one to OpenAI, and that was a fun day, but most of our success was Aligned around, um, in the beginning, just about helping the researchers get to the next level. I knew it wasn't very useful in its current state, but I also believe that in a few clicks, it could be really remarkable, and that belief system came from the interactions with all these amazing researchers, and it came from just seeing the incremental progress. At first, the papers were coming out every three months, and then, then papers today are coming out every day, right? So you could just monitor the archive papers, and I took an interest in learning about the progress of deep learning, and, and, and to the best of my ability, read these papers, and you could just see the progress happening, you know, in real time, exponentially in real time.

AI assessment note: “I wasn't involved in the founding of it, but I knew a lot of the people”

Q You believed that, like, kind of a version of the, was it the Jerry Sanders real men have fabs? Like, you need to do the whole stack. Like, you got to do everything, and that LSI Logic changed you.

A What LSI Logic did was, was, uh, realized that you can express Um, transistors and logical gates and chip functionality in high-level languages. That by raising the level of abstraction in what is now called high-level design, it was coined by Harvey Jones, who's on NVIDIA's board, and I met, met him way back in the early days of Synopsys. But, but during that time, there was this belief that you can express chip design in high-level languages. And by doing so, you could take advantage of optimizing compilers and optimization logic and, and, and tools, um, and, and be a lot more productive. That logic was so sensible to me, and I was 21 years old at the time, and I, I want to pursue that vision. Now, frankly, that, that idea happened in, in, um, uh, machine learning, it happened in, you know, software programming, and I want to see it happen in digital biology so that we can, we can think about, uh, biology in a much higher level language. Uh, probably a large language model, um, would be the, the way to make it, make it representable. That transition was so revolutionary, I thought that was the best thing that ever happened to the industry, and I was, I was really happy to be part of it, and I was at ground zero, and so, so I, I saw one industry, um, change, revolutionize another industry, and if not for LSI Logic doing the work that it did, uh, synopsis shortly after, then wh…

AI assessment note: “That logic was so sensible to me, and I was 21 years old”

Q Did you ever feel like, oh man, we might have invested ahead of the demand for machine learning since we're like a decade before the whole world is realizing it?

A I guess yes and no. You know, when we saw deep learning, when we saw AlexNet, And realized its incredible effectiveness in computer vision. We had the good sense, if you will, to go back to first principles and ask, you know, what is it about this thing that made it so successful? When a new software technology, a new algorithm comes along, and somehow leapfrogs 30 years of computer vision work, you have to take a step back and ask yourself, but why? And fundamentally, is it scalable? And if it's scalable, what other problems can it solve? And there were several observations that we made. The first observation, of course, is that if you have a whole lot of example data, you could teach this function To make predictions. Well, what we've basically done is discovered a universal function approximator, because the dimensionality could be as high as you wanted to be, and because each layer is trained one layer at a time, there's no reason why you can't make very, very deep neural networks. Okay, so now you just reason your way through, right? Okay, so now I go back to, 12 years ago. You could just imagine the reasoning I'm going through in my head that we've discovered a universal function approximator. In fact, we might have discovered with a couple more technologies, a universal computer that you can teach.

AI assessment note: “I guess yes and no. You know, when we saw deep learning, when we saw AlexNet”

Q Ok, so wait, wait, first question. Was that the plan all along? Like, when, when did you realize?

A I realized, I didn't learn about it until it was too late. We should have implemented all three too, yeah. But, but we built, we built, and so we had to make the best of it. That was really an extraordinary time. Remember, Revo one 20 was mv three. NV-one and NV-two were based on forward texture mapping, no triangles but curves, and it tessellated the curves, and because we were rendering higher level objects, we essentially avoided using Z-buffers, and we thought that that was going to be a good rendering approach, and turns out to have been completely the wrong answer. And so what Revo Run-Twenty-eight was, was a reset of our company. Now remember, at the time that we started the company in 1993, We were the only consumer, three D graphics company ever created, and we, we were focused on transforming the PC into an accelerated PC because at the time, Windows was really a software rendered system. And so anyways, Riva one, 28 was a reset of our company because by the time that we realized we had gone down the wrong road, Microsoft had already rolled out DirectX. It was fundamentally incompatible with Nvidia's architecture. 30 competitors have already shown up, even though we were the first company at the time that we were founded. So the world was a completely different place. The question about what to do as a company strategy, at that point, I would have said that we made a …

AI assessment note: “I realized, I didn't learn about it until it was too late.”

Q that, I mean, I know how meaningful that is in any company, but for you, given that, I feel like the NVIDIA journey is, um, particularly amplified on these dimensions, right? And like, you know, you went through two, two, if not three, like 80% plus drawdowns in the public markets to have investors who've stuck with you from day one through that must be just like, So much support.

A Yeah, yeah, it is incredible, and you hate that any of that stuff happened, and, and most of you, you know, most of it is, is out of your control, but, you know, 80% fall, it, it, it's an extraordinary thing, no matter how you look at it, and I forget exactly, but I mean, we, we traded down at about a couple of Two, three billion dollars in market value for a while because of the decision we made and going into CUDA and all that work, and your belief system has to be really, really strong. You know, you have to really, really believe it and really, really want it. Otherwise, it's just too much to endure. I mean, because, you know, everybody's questioning you and employees aren't questioning you, but employees have questions.

AI assessment note: “Yeah, yeah, it is incredible, and you hate that any of that stuff happened”

Q You can't really imagine that, whereas that happens here. Are there other companies, either current or historically, that you look up to, admire, maybe took some of this inspiration from?

A In the last 30 years, I've read my fair share of business books, and as in everything you read, you, you're supposed to, you're supposed to, to first of all, enjoy it, right? Enjoy it, be inspired by it, but not to adopt it. That's not the whole point of these books. The whole point of these books is to share their experiences, and, and you, you're supposed to ask, you know, what does it mean to me in my world, and what does it mean to me in the context of what I'm going through? What does this mean to me and the environment that I'm in? And what does this mean to me and what I'm trying to achieve? And what does this mean to NVIDIA and the age of our company and the capability of our company? And so you're supposed to ask yourself, what does it mean to you? And then from that point, being informed by all these different things that we're learning, uh, we're supposed to come up with our own strategies. You know, what I just described is kind of how I go about everything. You're supposed to be inspired and learn from every, everybody else. And, and the education's free, you know? When somebody talks about a new product, you're supposed to go listen to it. You're not supposed to ignore it. You're supposed to go learn from it. And it could be a competitor. It could be a adjacent industry. It could be nothing to do with us. The more we learn from what's happening out in the world, t…

AI assessment note: “You're supposed to be inspired and learn from every, everybody else.”