Aug 5, 2024 · 36m · a16z
How Waymo Is Using GenAI to Build a Better Driver
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of a16z's 'AI Revolution' series, Waymo co-CEO Dmitry Dolgov joins David George to discuss the technological evolution of autonomous vehicles, from the 2007 DARPA Urban Challenge to integrating modern foundation models, vision-language architectures, and custom sensor stacks. Dolgov highlights how Waymo leverages closed-loop simulation, scaling laws, and rigorous safety engineering to successfully deploy driverless commercial ride-hailing services.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
speaking balance: gold is the host, purple is the guest (3 minute bins)
Dmitry directly pushes back on David's premise of end-to-end AI replacing legacy approaches, labeling it a weird dichotomy and insisting that full autonomy requires deep learning plus extensive safety engineering.
Hardest push from the host ▶ 14:42 Challenging end-to-end taglinesDavid explicitly pushes Dmitry with an intentionally simplifying premise about end-to-end driving taglines that he acknowledges might drive Dmitry crazy, challenging him to defend Waymo's hybrid architecture.
Biggest teaching moment ▶ 28:06 Sensor physics breakdownDmitry educates David on why LiDAR cannot be omitted for full autonomy, breaking down the distinct physical properties and redundancy advantages of active LiDAR, imaging radar Doppler velocity, and high-res cameras.
The host holds their own ▶ 20:26 Urban pick-up edge casesDavid demonstrates strong domain knowledge by offering specific, accurate edge cases about dense urban pick-ups—such as opening garage doors and driveway blocking—prompting Dmitry to confirm he hit the exact subtleties of the problem.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Revolution Series Title Graphic | 2 | 3 | 0 | 0 | David George asks Dmitry to reflect on his early DARPA challenge days and career evolution. Dmitry warmly explains the 2007 Urban Challenge setup and how it sparked his work leading to Google Self-Driving Car Project and Waymo. | |
| Integration of ConvNets, Transformers, and GenAI in Autonomous Driving | 3 | 4 | 1 | 1 | David asks about layering generative AI onto traditional machine learning, prompting Dmitry to gently reframe the scope by detailing the chronological integration of ConvNets, Transformers, and VLMs. | |
| Simulation, Synthetic Data, and Closed-Loop Testing | 4 | 4 | 0 | 1 | David highlights the debate around synthetic data utility, and Dmitry details how closed-loop simulation generates long-tail edge cases and statistical realism for training. | |
| Applying Scaling Laws to Autonomous Systems and Onboard Compute Constraints | 5 | 5 | 3 | 3 | David presents the debate between rules-based edge-case engineering and pure end-to-end AI, prompting Dmitry to reject the dichotomy and explain why end-to-end models alone cannot solve full autonomy. | |
| Real-World Deployment, Weather Conditions, and Pick-Up/Drop-Off Challenges | 5 | 2 | 0 | 1 | Dmitry discusses pick-up and drop-off challenges, and David demonstrates solid domain understanding by listing specific urban friction points like garage doors and driveway blocking. | |
| Safety Record, Swiss Re Study, and Safety Validation Framework | 5 | 4 | 0 | 1 | David cites recent 15 million mile safety data, while Dmitry enriches the discussion with joint Swiss Re study findings and Waymo's attentive-human benchmark model. | |
| Generalizable Driver Vision, Go-To-Market Strategies, and Partnerships | 3 | 3 | 0 | 0 | David asks about future commercial dynamics, prompting Dmitry to detail Waymo's generalizable driver vision across ride-hailing, trucking, and Uber app integrations. | |
| Hardware Modalities: LiDAR, Radar, and Cameras | 3 | 5 | 0 | 0 | David asks whether LiDAR will remain necessary, leading Dmitry to explain the complementary physics of cameras, LiDAR, and imaging radar for full autonomy redundancy. | |
| Why Driverless Autonomy is Uniquely Hard Compared to LLMs | 5 | 4 | 0 | 1 | David contrasts the rapid commoditization of digital LLMs with the consolidation in AVs, which Dmitry attributes to physical world noise, safety stakes, and real-time millisecond latency constraints. | |
| Humbling Early Memories and Career Advice for Builders | 2 | 1 | 0 | 0 | David prompts Dmitry for early memories, leading to an amusing 2009 anecdote about an AV navigating around debris falling from a dump truck, followed by career advice for young builders. |