Jun 26, 2023 · 1h 0m · a16z
Are Autonomous Vehicles Finally Here? Buckle up!
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the a16z Podcast, host Catherine takes a driverless ride across San Francisco with Waymo Chief Product Officer Saswat Panigrahi to experience Level 4 autonomous vehicle technology firsthand. Their conversation explores the technical architecture, multi-sensor hardware, machine learning models, safety frameworks, and societal impacts driving the future of commercial autonomous mobility.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The host as informed peer 3.1 Guest teaching 3.4 Guest disagreement 0.2 The host pushing back 0.5
speaking balance: gold is the host, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 15:15 Rejecting Lidar vs Camera Debate Framing
Saswat forcefully rejects the host's premise that choosing between Lidar and camera vision is a binary debate, calling it an ideological trap and comparing it to arguing over light wavelengths.
Hardest push from the host ▶ 36:25 Challenging the Term Pushback for RegulatorsSaswat pushes back on Catherine's characterization of regulatory challenges, arguing that regulators and Waymo share the exact same core objective of public safety.
Biggest teaching moment ▶ 6:13 Explaining Level 3 vs Level 4 DifferenceSaswat corrects Catherine's understanding of autonomy levels, clarifying that the jump from Level 3 to Level 4 is an enormous paradigm shift comparable to the difference between driving and flying.
The host holds their own ▶ 49:20 Citing Psychological Principles on Driving AbilityHost Catherine demonstrates strong subject familiarity by bringing up the fundamental attribution error to explain why human drivers incorrectly rate their skills above statistical averages.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The host as informed peer | Guest teaching | Guest disagreement | The host pushing back | Why |
|---|---|---|---|---|---|---|
| Legal Disclosures and Official a16z Podcast Opening | 1 | 0 | 0 | 0 | This segment consists of official disclosures, intro narration, and setting up the audio inside the Waymo car. Because it is a monologue intro, all scores remain minimal. | |
| Initial Ride Reactions and Waymo Historical Development Milestones | 2 | 4 | 1 | 1 | Catherine asks Saswat to map out the 5 levels of autonomy and inquires about level 3 vs level 4. Saswat politely corrects her understanding, explaining that the shift from level 3 to 4 is as vast as driving versus flying because human intervention is completely removed. | |
| Waymo Full-Stack Hardware and Software Engineering Strategy | 2 | 3 | 0 | 0 | Catherine asks about the historical barriers to level 4 autonomy. Saswat explains why Waymo was forced to take a full-stack approach, developing custom hardware, sensors, software, and simulation in-house rather than waiting on third parties. | |
| Comprehensive Breakdown of Lidar, Camera, and Radar Sensors | 2 | 4 | 0 | 0 | Catherine asks Saswat to break down the physical sensors visible on the vehicle exterior. Saswat educates her on how lidar, high-resolution cameras, and radar complement each other across different ranges and weather conditions. | |
| Machine Learning, Behavior Prediction, and Pedestrian Intent Recognition | 3 | 4 | 0 | 0 | The conversation covers pedestrian intent and machine learning models. Saswat details how subtle bodily cues like gait and hand movement are parsed to transition the vehicle from over-conservative stopping to smooth assertion. | |
| Evaluating Lidar Versus Camera Systems and Cost Trajectories | 5 | 5 | 2 | 2 | Catherine introduces the industry debate over Lidar vs camera-only Vision systems and asks about build vs buy dynamics. Saswat rejects the premise of an ideological debate, arguing that preferring one sensor over another is like picking arbitrary light wavelengths rather than sound engineering. | |
| Building Competitive Moats, Scaling, and AI Model Generalization | 4 | 3 | 0 | 1 | Catherine brings up economic moats and data advantages in autonomous driving. Saswat explains knowledge curves and simulation flywheels, supported by a live road incident where the vehicle seamlessly reacts to a human driver running a red light. | |
| Simulation Infrastructure and Testing Weather and Traffic Hazards | 3 | 4 | 0 | 0 | Catherine asks whether driving models are fine-tuned per scenario or run on a single aggregate architecture. Saswat explains the multi-layered deep learning stack and highlights Google's compute infrastructure as a key technical moat. | |
| Multi-Tiered Safety Frameworks and Engineering Real Passenger Trust | 3 | 4 | 0 | 0 | Catherine prompts a discussion on how safety is evaluated by regulators, developers, and riders. Saswat details four distinct layers of safety ranging from collision statistical modeling to passenger perceived trust. | |
| Navigating Complex Traffic and Cyclists | 2 | 3 | 0 | 0 | The ride encounters a large group of bicyclists. Saswat points out how the system tracks handlebar angles in real time to predict cyclist direction and maintain ride progress without overly aggressive stopping. | |
| Autonomous Perception vs. Human Driver Limits | 4 | 3 | 1 | 2 | Catherine asks how Waymo manages regulatory pushback across various cities. Saswat gently resists the framing of regulatory 'pushback', arguing that regulators and autonomous operators share the foundational goal of reducing traffic fatalities. | |
| In-Cabin Screen Design and Rider Comfort | 3 | 3 | 0 | 0 | Saswat discusses in-cabin user experience and screen visualization decisions. He notes that early designs displayed too much raw perception data, which increased passenger anxiety rather than reducing it. | |
| Rider Feedback Flywheel and Contextual Drop-Off Rules | 3 | 4 | 0 | 0 | Catherine asks about user feedback loops. Saswat explains complex pick-up and drop-off contextual rules, contrasting narrow residential streets with busy commercial thoroughfares. | |
| Reimagining Car Design for a Driverless Future | 4 | 3 | 0 | 1 | Catherine asks about future vehicle design paradigms, such as eliminating steering wheels. Saswat discusses current regulatory requirements and custom platform collaborations with OEMs like Geely. | |
| Arriving at the Painted Ladies and Neighborhood Reactions | 2 | 3 | 0 | 0 | The vehicle arrives at the Painted Ladies. Saswat points out pedestrian reactions and explains how Waymo selected 20 diverse cities during early development to establish generalizable driving capabilities. | |
| Overcoming Rider Apprehension and Driver Perception | 5 | 3 | 0 | 1 | Catherine cites the psychological concept of fundamental attribution error to explain why most humans unrealistically view themselves as superior drivers. Saswat expands on broader economic and urban impacts, such as converting parking real estate into public spaces. | |
| Autonomous Trucking and Real-Time Construction Detection | 4 | 4 | 0 | 0 | Catherine shares personal experience driving long-haul routes at night among trucks. Saswat illustrates how the perception stack navigates unpredictable construction zones by parsing written signs and pylon arrays in real time. |
Statements from this episode (36)
Debating LiDAR versus cameras is just choosing between wavelengths, says Waymo CPO
“Saying you love lidars and hate cameras or vice versa is saying you love one wavelength versus the other wavelength, right? It's not a fundamental thing.”
Panigrahi: Waymo rides require no front-seat human driver or takeover
“Fully autonomous as you can see, nobody in the front seat, no expectation of a human to take over.”
Panigrahi: Waymo suspends autonomous operations in heavy snow
“So, we are in that level four with a certain scope, So right now, if we were to begin heavily snowing in SF, which it has, believe it or not last season, there was a little bit of snow we wouldn't operate it.”
Level 3 versus Level 4 autonomy is like driving versus flying
“No, no, I would say a vast difference even between level three and level four. Huge. It's almost a difference between driving and flying, I would say.”
Panigrahi: Waymo vehicles automatically pull over if a rider interferes
“So if you did, the car would say I'm being interfered with. I'm a fully autonomous car. I'm not supposed to be interfered with in this manner. So it'll, it'll pull over, basically.”
Autonomous vehicles require full-stack hardware and software to maintain iteration speed
“And the technology, and it really required the full stack, right? We build the hardware, we build the software, because if you built just the software and waited for somebody else to deliver the hardware, the speed of learning, the speed of iteration that was …”
Panigrahi: Waymo vehicles can see 360 degrees up to three football fields away
“This car with those appendages, as you mentioned, can see three football fields away, 360 degree, and it's getting a snapshot multiple times a second.”
Radar sensors allow Waymo autonomous vehicles to see around corners
“The radars can almost see around corners. Even when the laser and the camera or our human eyes can't.”
Autonomous vehicles must analyze pedestrian gait and body language to predict intent
“So for example, for that pedestrian, ah, in addition to seeing that they're there, acknowledging, which was this problem one, you got to look at even their gait, their hand movement, their leg movement to anticipate, are they about to take motion?”
Waymo could detect pedestrians in 2019, but struggled with over-conservatism
“And if you're always conservative, which was, you know, say four years ago, it was not like we couldn't detect the pedestrians, we could totally detect them. It was this nuance of, are we being over conservative, assuming they may jump in, hence let's not move…”
Panigrahi: Multi-sensor suites outperform camera-only autonomous driving systems
“Does the combination of these sensors position you better than individual. The answer is yes. We can show you that there are situations in which camera will be insufficient.”
Panigrahi: Waymo dramatically reduced LiDAR hardware costs over two years
“The amount we have been able to cost down these LiDARs in the last two years is incredible.”
Panigrahi: Off-the-shelf LiDAR and radar were unsuited for autonomous driving
“What we found is that the absolute best LiDAR out there, absolute best radar out there was not optimized for the task of autonomous driving. And hence you know we had to build it.”
Waymo has logged over 20 million real-world testing miles
“By having driven twenty million plus miles in testing, by having done billions of miles of simulation, we become aware of problem spaces that others may not have discovered yet.”
Solving San Francisco and Phoenix traffic allows autonomous driving to generalize globally
“Pretty much every good weather city is like a linear combination of those two things, right? So in Los Angeles, Exactly. So you go to West Hollywood, you're much more like a SF style driving. Lots of pedestrians, cyclists, and so on. You go to the LA's you kno…”
Saswat Panigrahi: Waymo uses hybrid specialized deep models, not one single algorithm
“It's definitely many, many deep models. Some very general, extremely deep learning models, and some specialized models to make them really good at some very hard tasks.”
Saswat Panigrahi: Generative AI and deep models eliminate manual hand-tuning
“One of the powerful things that deep models are telling us As well as Generative AI is telling us is that you actually don't need to hand tune every single thing. It works.”
Google's early machine learning infrastructure investments created Waymo's competitive moat
“Really the breakthrough engineering that you sometimes need in AI is just having the raw infrastructure to intake all this data. The amount of data you have to learn to handle to build a really, you know, well-learned algorithm is pretty hard, and that's where…”
Panigrahi: Waymo vehicles use audio microphones to locate emergency sirens
“There are microphones that can not only hear that there's a siren, but also you know, point at where the siren is coming from.”
Waymo vehicles detect cyclist handlebar orientation to predict their movement intent
“Because we can see that their handles are turning leftward, so we can understand that they're likely going to go that way.”
Waymo re-simulations successfully avoided every fatal crash in Phoenix's East Valley
“We took every fatal crash that had occurred and re-simulated and showed that Wemo could avoid that.”
Panigrahi: Waymo was first to cross one million fully driverless miles
“We were the first company ever to cross one million fully autonomous miles.”
Waymo logged zero injury collisions across its first million fully autonomous miles
“In that, we published our full crash text. And there was not a single collision with injury. Not a single one. And only two of them would, you know, meet the standards of a reportability called CISS.”
Panigrahi: Waymo warns passengers about approaching cyclists to prevent dooring
“And the reason, by the way, ah, we're switching a little bit from safety to user experience design, the reason it told you cyclists approaching is what we know is even when the car is stopped, what happens is people open the door, and the cyclist was like, I m…”
Waymo has driven millions of autonomous miles and billions in simulation
“And by the way, the two million thing that you mentioned, that's just the fully autonomous miles. We have billions in simulation.”
Granular passenger screen displays cause riders to fixate on technical details
“So we tried to experiment with how much detail we put in here, and there are folks who, when in the early days we had a version here that would show a lot more detail, and they would engage like this, right? In fact, they would look more inside than outside th…”
Panigrahi: Autonomous drop-offs require contextual rules over static logic
“It depends on the context. If you're on a busy street like that, and somebody's going to a business, they would like to be dropped very likely in front of the business they're going to. Whereas if they are getting out of home, going to work, and it's a narrow …”
Waymo CPO: Regulations mandate steering wheels on larger autonomous fleets
“Specifically about the steering wheel block. Currently by regulations beyond a certain number of vehicles and so on.”
Panigrahi: Waymo designed its next-gen autonomous vehicle platform with Geely
“Our next generation vehicle that we did design with C, VT, and Geely is a pretty powerful platform taught with the rider in mind.”
Panigrahi: Waymo AVs drive defensively without assuming drivers obey lights
“It thinks about many small details. Where is it positioning itself? How much gap is it leaving? What's its velocity so that it has the greatest optionality if somebody were to behave recklessly. So when we are crossing that sign, we're not assuming that everyb…”
Panigrahi: Waymo operates fully autonomous rides and airport pickups in Phoenix
“And in Phoenix, we'll take you to down to Scottsdale, and when you land at the airport, we will pick you up at the airport. That's where we are in the journey. We're fully autonomous where we need to be.”
Waymo CPO claims company has achieved full autonomy where needed
“We're fully autonomous where we need to be.”
Panigrahi: Waymo vehicles programmatically exercise heightened caution around children
“And it's more cautious, because it understands that kids can jump out more erratically than others can.”
Driverless vehicles will be mainstream before today's children reach adulthood
“And by the time he grows up, this will be the more common route.”
Panigrahi: Waymo tested vehicles in 20 diverse environments including Death Valley and Tahoe
“We have tested, for example, in Miami as well for the heavy rain. We have tested in Death Valley for extreme temperatures. We have, ah, tested in Tahoe for snow. So there was a testing phase in which we went to 20 cities just to make sure with enough of a dive…”