Dolgov: End-to-end AI models alone cannot achieve full autonomous driving
Dmitry Dolgov · How Waymo Is Using GenAI to Build a Better Driver · Aug 5, 2024 · at 16:58
Dmitry Dolgov is the co-CEO of Waymo. On The a16z Podcast, he addresses the industry debate surrounding end-to-end AI architectures versus hybrid AV stack engineering.
0:00 / 0:29exact quote · 29.6s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“You can take, you know, an end-to-end model, go from sensor to, you know, trajectories or actuation. You know, typically you don't build them in one stage, you build them in stages, but, you know, you can do, like, backprop through the whole thing, so, you know, the concept is very, very valid. You can, you know, combine it in, you know, with a VLM, and then, you know, you add closed loop simulation, some sort, and, you know, you're off to the races. You can have a great demo, like, almost out of the box. You can have, you know, an ADESP or a driver assist system. But that's not enough to go all the way to full autonomy.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dmitry Dolgov
Dolgov: Waymo will continue using LiDAR, radar, and cameras together
“So yeah, I think the trend, you know, for us that we'll, you know, using all three modalities just, you know, makes a lot of sense.”
Dolgov: Distilling huge AI models beats training small models directly
“Where you're much better off training a huge model and then distilling it into a smaller model than just training small models.”
Dolgov: Waymo driverless cars cut injury crashes by 3.5x
“And I think 3.5 acts as the reduction in injury. And that's about two X reduction in the police reportable kind of lower severity incidents.”
Dolgov: Swiss Re study showed zero bodily injury claims for Waymo
“We published a joint study with a Swiss RE. Which is, I think, the largest global reinsurer in the world, and the way they look at it is, you know, who contributed to an event, and there we saw, ah, like the same theme, but the numbers were, ah, very strong, t…”
Dolgov: Waymo is combining autonomous driving AI with vision-language models
“So that, that's what we've been very focused on that Waymo most recently is taking kind of the AI backbone and all of the AI, the Waymo AI that is over the years we've built up that is really proficient at this task of autonomous driving and combining it with …”
Dolgov: Building AV drivers requires iteratively modeling simulation environments and actors
“To build a good driver, you need to have a very good simulator, but to have a good simulator, you actually have to build models of, like, realistic pedestrians and cyclists and drivers, right? So it's, you know, you kind of do that iteratively.”