Dolgov: Waymo is combining autonomous driving AI with vision-language models
Dmitry Dolgov · How Waymo Is Using GenAI to Build a Better Driver · Aug 5, 2024 · at 8:44
Dmitry Dolgov, co-CEO of Waymo, explains how Waymo is evolving its tech stack by combining domain-specific autonomous driving AI with modern generative vision-language models.
0:00 / 0:21exact quote · 21.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So that, that's what we've been very focused on that Waymo most recently is taking kind of the AI backbone and all of the AI, the Waymo AI that is over the years we've built up that is really proficient at this task of autonomous driving and combining it with kind of the general world knowledge and understanding of these, you know, VLMs.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dmitry Dolgov
Dolgov: End-to-end AI models alone cannot achieve full autonomous driving
“You can take, you know, an end-to-end model, go from sensor to, you know, trajectories or actuation. You know, typically you don't build them in one stage, you build them in stages, but, you know, you can do, like, backprop through the whole thing, so, you kno…”
Dolgov: Waymo will continue using LiDAR, radar, and cameras together
“So yeah, I think the trend, you know, for us that we'll, you know, using all three modalities just, you know, makes a lot of sense.”
Dolgov: Distilling huge AI models beats training small models directly
“Where you're much better off training a huge model and then distilling it into a smaller model than just training small models.”
Dolgov: Waymo driverless cars cut injury crashes by 3.5x
“And I think 3.5 acts as the reduction in injury. And that's about two X reduction in the police reportable kind of lower severity incidents.”
Dolgov: Swiss Re study showed zero bodily injury claims for Waymo
“We published a joint study with a Swiss RE. Which is, I think, the largest global reinsurer in the world, and the way they look at it is, you know, who contributed to an event, and there we saw, ah, like the same theme, but the numbers were, ah, very strong, t…”
Dolgov: Building AV drivers requires iteratively modeling simulation environments and actors
“To build a good driver, you need to have a very good simulator, but to have a good simulator, you actually have to build models of, like, realistic pedestrians and cyclists and drivers, right? So it's, you know, you kind of do that iteratively.”