The Exchanges

Q Yeah. I was gonna ask you what the path forward is given that we know that, okay, this is something that's these models are struggling with. So you think that this is the path forward that the way that you just described?

A You look at the innovation that's taking place from, from players out there. Companies like wave in the UK have put together a really amazing, uh, underlying technology that lets the front facing camera be interpreted in a way that the system can communicate to the occupants of the vehicle, what's going on outside the car. And also use that then as a way to determine what is the car going to do? How is it going to steer, accelerate, or brake? So it can interpret that video feed and explain that there's somebody jaywalking, or somebody ran a red light, or there's a child waiting to cross, or whatever it may be. So this generative AI approach is really, I think, going to accelerate the adoption. Very recently at a conference called CVPR, so that's Computer Vision Pattern Recognition. It takes place annually. Uh, last month it was in Seattle, Washington, and there was a competition that they held, so it's a research, uh, conference, over 400 entries into this autonomous driving challenge, and it was basically looking at sensor data and trying to predict the best trajectory for the vehicle moving into the future. NVIDIA submitted and won the challenge. Our research team had developed a new large language model, basically end to end training of that sensor data system for then controlling the vehicle. And so over 400 entries, NVIDIA came out on top with this new large language model…

AI assessment note: “this generative AI approach is really, I think, going to accelerate the adoption”

Q to me that NVIDIA is developing a lot of this technology. Like I went to your automotive section on the website and was like, wow, there's a tremendous amount of models coming out of NVIDIA. I thought it was largely the car makers, like the Waymos or the Teslas that are developing the autonomous technology. So how involved is NVIDIA in developing these models itself? And then who's the customers?

A That's a great question. Um, we work with hundreds of automakers, truck makers, robo taxi companies, um, software startups, the sensor companies, the mapping companies. It really is quite an ecosystem that we've built. We're not creating the vehicles, but we work with those manufacturers, and so we offer the compute hardware. That's our drive platform, so that's the brain that goes inside the car. Our drive OS is the safe operating system that's part of that package. We have a lot of different middleware and libraries that they can use to develop their applications, algorithms, the neural networks. That application layer, though, is generally built by our customers, so Mercedes-Benz or Jaguar, Land Rover, Volvo, Neo in China, and so they can pick whatever parts of the software stack they want, and in many cases, our customers are taking the whole stack and they're developing Um, some of their own algorithms as well. So there might be, uh, a pedestrian detection algorithm from Mercedes running along a pedestrian detection algorithm from NVIDIA. And, uh, and we collaborate on that. So starting, um, through the end of this year, the introduction of the, the new CLA, uh, it's already been announced from Mercedes. That's the new Mercedes model, the C-Class. And, um, So every Mercedes will be built on NVIDIA drive with the software that we've developed and rolled out by NVIDIA. So it…

AI assessment note: “we work with hundreds of automakers... Mercedes-Benz or Jaguar, Land Rover, Volvo”

Q So then how important is simulation and training? Anything having to do with autonomous driving?

A That simulation plays a really, really big role in ensuring the safety of the system. There's really no way that, first of all, driving around collecting data, you rarely are going to see the dangerous scenarios, the hazards, the things that, you know, very rarely occur. You're not going to capture them on your data collection. So we need to use simulation and really what we call synthetic data generation to create those kinds of scenarios. So we can Create fake potential hazards, things falling off of trucks and people running across the street at night, um, whatever it may be, somebody running a red light. And so we can create that data to augment the real data for training the AI. And then we can actually simulate all these dangerous scenarios to ensure that the system will do the right thing. And the benefit of using simulation too, it's repeatable. So we can adjust the software and test something that maybe didn't pass a month ago, but we can run it through the same scenario and say, oh yeah, we fixed that. Um, there may be situations, you know, it often happens that the sensors, um, are blinded by the sun, right? As it's setting, right? The sun is like coming right into the, the eyes of the, the car, into the driver's eyes, into the camera's eyes. And so you only have a few minutes a day where you can actually capture that data as well as test. And so in simulation, it ca…

AI assessment note: “simulation plays a really, really big role in ensuring the safety of the system”

Q specifically like, um, going into the technical details, but you mentioned that we have autonomy already. So why is it taking us so long to, and I know so long as relative, but so long to, uh, have that spread from one company that does it really well. To every car through the economy. Is it a safety thing? Is it a cost thing? Like, what is the roadblock now?

A Well, for us, and I believe a lot of our partners, safety is the primary concern. I think if you look back to 2016, when a lot of predictions were made, everyone was talking about 2020 was the year. And so from a compute standpoint, from a software development standpoint, that really looked realistic. I think everyone underestimated the true complexity of being able to make sure you get it right. Virtually all the time, and that's what's really challenging. So the basics are easy when you can drive down the freeway. Cars are all going in the same direction. There's no pedestrians. There's good lane markings. That's really a solved problem. But as soon as you really introduce the complexity and the, the anomalies that come about from human behavior, people either falling asleep on the roads or driving recklessly or being impatient or road rage or whatever it is, Um, it's really hard to predict that and creates hazards for the self-driving car. So I think what we're doing is we're seeing a whole new wave of innovation now. Part of it's based on, you know, same fundamental technology from chat GPT, these large language models, more of an end-to-end system that's able to look holistically at the whole environment around the car, uh, and be able to anticipate and predict what other drivers will do and understand how to react. Just like ChatGPT, you can say anything now and it knows …

AI assessment note: “Well, for us, and I believe a lot of our partners, safety is the primary concern.”

Q Right. Now talking again about what NVIDIA is doing internally, it's pretty wild. So you're actually simulating collisions. Is this something that you do in that sort of world that you talked about where you go through these?

A What we try to do is, um, create the scenarios. It's less about, um, you know, creating accidents, but creating the scenarios to ensure that the systems will do have a safe outcome. Um, there's no way to ensure that There's zero accidents in the world, right? There's just, there's, there's always can be crazy stuff that's going to happen on the road and no human driver could, could avoid something falling right in front of the car or somebody getting pushed in front of a vehicle or something like that. But what we want to do is be able to anticipate all that and be able to avoid it or mitigate, um, what would happen, um, in one of these hazardous scenarios. One of the things we're able to do actually is, um, record drives that we're taking and then use that as input to create a huge range of different scenarios, permutations on that and test the software. And so we can actually capture cars in a scene and make any one of those cars in the scene the autonomous car and see how it would behave. So we're building a massive database of scenarios and ways to test and validate that the technology Is good. The other thing that we can do is we can take accident reports and now using, um, these large language models, we can input these, uh, these accident reports and be able to create scenarios from a text input explaining what happened or, um, if there's a map or something like that.

AI assessment note: “What we try to do is, um, create the scenarios.”

Q talk to each other. You're on two different projects and maybe, you know, wink, wink. That's why the automotive project within Apple failed. And then other companies incentivize silos just by the incentives in terms of like your performance review. If you were, you know, coming up short, even though you were collaborating, you came up short, you get this grade, you don't advance. So how does Nvidia address this?

A That's the, those things you describe are not the culture of our company. I think one of the, the, um, principles that were founded on is one team. It's all about Nvidia first. And so the individual group, Kind of come second, and in fact, the notion of the group is kind of dynamic, and we really don't have much of an org chart in the company. Jensen says the mission is the boss, and so we have these virtual teams. There's a lot of cross-functional work that goes on. People have different roles and responsibilities and might be working on a variety of different things, and so it's really all about how does, what's the best thing for the company, and working across groups is really rewarded, and in part of the way that Just the culture is we want to help each other and the whole company succeeds as opposed to, hey, this is my thing. I own this. I'm just going to focus on this. So it really is, is part of the, just the culture of the company and embraced throughout. And Jensen is constantly looking at, okay, if there's, if he finds two different groups that are doing related things, he's like, you guys get together and figure it out together. We don't need to have two separate, uh, Uh, programs going on here, but let's, let's pick the best. And so there really is a huge collaboration that goes on throughout the company.

AI assessment note: “one of the, the, um, principles that were founded on is one team.”