StarZero All shows Try the podcast clipper →

Nov 13, 2019 · 22m · mad

Human Intuition for Machines // Sam Anthony, Perceptive Automata (FirstMark's Data Driven NYC)

Sam Anthony · 18m spoken Matt Turck · 20s spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

At FirstMark's Data Driven NYC, Sam Anthony, CTO and Co-Founder of Perceptive Automata, presents 'Human Intuition for Machines,' demonstrating how psychophysics and neuroscience can teach autonomous vehicles to interpret human intent and navigate complex urban environments safely.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 1.7% of the talking time here. How this is scored →

Matt as informed peer 0.6 Guest teaching 3.8 Guest disagreement 1.4 Matt pushing back 0.4
05100:0010:0020:000:14–5:06 · Matt as informed peer 0/10 The Physics-Only AV Limitation Sam Anthony opens his presentation by challenging the physics-only approach used by current autonomous vehicles. Because this is a monologue presentation, host metrics are zero. Sam highlights Moravec's paradox to explain why human social intuition is hard for machines.5:06–7:53 · Matt as informed peer 0/10 Subtle Cues and Industry Consensus Sam details subtle physical cues like bag-holding tension that indicate pedestrian intent, noting industry leaders agree this is the main hurdle. He references reports of pedestrians punching timid self-driving cars to underscore brand risk. As a monologue, host scores remain zero.7:53–12:10 · Matt as informed peer 0/10 Perceptive Automata's Methodology and Data Pipeline Sam explains Perceptive Automata's methodology of running human behavioral science experiments at scale to train machine learning models. He emphasizes answering questions as a virtual crowd vote rather than building general AI. Host scores remain zero for this monologue section.12:10–16:08 · Matt as informed peer 0/10 Real-World Computer Vision Demonstration Sam walks through real-world visual demonstrations showing intention and awareness models in action. He highlights an edge case of a jaywalker at a cab stand who breaks standard physics and historical rules, but whose intent is clear to human intuition. Host scores are zero during the presentation.16:08–22:58 · Matt as informed peer 3/10 Fireside Q&A with Matt Turck Host Matt Turck opens the Q&A by asking about company background and probing how they know they are ready for commercial deployment. Audience members follow up with questions on facial expressions, local city variations, and unpredictable actors like children or drunk people. Sam responds collaboratively and clarifies how ambiguity is modeled as a safety signal.0:14–5:06 · Guest teaching 3/10 The Physics-Only AV Limitation Sam Anthony opens his presentation by challenging the physics-only approach used by current autonomous vehicles. Because this is a monologue presentation, host metrics are zero. Sam highlights Moravec's paradox to explain why human social intuition is hard for machines.5:06–7:53 · Guest teaching 4/10 Subtle Cues and Industry Consensus Sam details subtle physical cues like bag-holding tension that indicate pedestrian intent, noting industry leaders agree this is the main hurdle. He references reports of pedestrians punching timid self-driving cars to underscore brand risk. As a monologue, host scores remain zero.7:53–12:10 · Guest teaching 4/10 Perceptive Automata's Methodology and Data Pipeline Sam explains Perceptive Automata's methodology of running human behavioral science experiments at scale to train machine learning models. He emphasizes answering questions as a virtual crowd vote rather than building general AI. Host scores remain zero for this monologue section.12:10–16:08 · Guest teaching 4/10 Real-World Computer Vision Demonstration Sam walks through real-world visual demonstrations showing intention and awareness models in action. He highlights an edge case of a jaywalker at a cab stand who breaks standard physics and historical rules, but whose intent is clear to human intuition. Host scores are zero during the presentation.16:08–22:58 · Guest teaching 4/10 Fireside Q&A with Matt Turck Host Matt Turck opens the Q&A by asking about company background and probing how they know they are ready for commercial deployment. Audience members follow up with questions on facial expressions, local city variations, and unpredictable actors like children or drunk people. Sam responds collaboratively and clarifies how ambiguity is modeled as a safety signal.0:14–5:06 · Guest disagreement 2/10 The Physics-Only AV Limitation Sam Anthony opens his presentation by challenging the physics-only approach used by current autonomous vehicles. Because this is a monologue presentation, host metrics are zero. Sam highlights Moravec's paradox to explain why human social intuition is hard for machines.5:06–7:53 · Guest disagreement 2/10 Subtle Cues and Industry Consensus Sam details subtle physical cues like bag-holding tension that indicate pedestrian intent, noting industry leaders agree this is the main hurdle. He references reports of pedestrians punching timid self-driving cars to underscore brand risk. As a monologue, host scores remain zero.7:53–12:10 · Guest disagreement 1/10 Perceptive Automata's Methodology and Data Pipeline Sam explains Perceptive Automata's methodology of running human behavioral science experiments at scale to train machine learning models. He emphasizes answering questions as a virtual crowd vote rather than building general AI. Host scores remain zero for this monologue section.12:10–16:08 · Guest disagreement 1/10 Real-World Computer Vision Demonstration Sam walks through real-world visual demonstrations showing intention and awareness models in action. He highlights an edge case of a jaywalker at a cab stand who breaks standard physics and historical rules, but whose intent is clear to human intuition. Host scores are zero during the presentation.16:08–22:58 · Guest disagreement 1/10 Fireside Q&A with Matt Turck Host Matt Turck opens the Q&A by asking about company background and probing how they know they are ready for commercial deployment. Audience members follow up with questions on facial expressions, local city variations, and unpredictable actors like children or drunk people. Sam responds collaboratively and clarifies how ambiguity is modeled as a safety signal.0:14–5:06 · Matt pushing back 0/10 The Physics-Only AV Limitation Sam Anthony opens his presentation by challenging the physics-only approach used by current autonomous vehicles. Because this is a monologue presentation, host metrics are zero. Sam highlights Moravec's paradox to explain why human social intuition is hard for machines.5:06–7:53 · Matt pushing back 0/10 Subtle Cues and Industry Consensus Sam details subtle physical cues like bag-holding tension that indicate pedestrian intent, noting industry leaders agree this is the main hurdle. He references reports of pedestrians punching timid self-driving cars to underscore brand risk. As a monologue, host scores remain zero.7:53–12:10 · Matt pushing back 0/10 Perceptive Automata's Methodology and Data Pipeline Sam explains Perceptive Automata's methodology of running human behavioral science experiments at scale to train machine learning models. He emphasizes answering questions as a virtual crowd vote rather than building general AI. Host scores remain zero for this monologue section.12:10–16:08 · Matt pushing back 0/10 Real-World Computer Vision Demonstration Sam walks through real-world visual demonstrations showing intention and awareness models in action. He highlights an edge case of a jaywalker at a cab stand who breaks standard physics and historical rules, but whose intent is clear to human intuition. Host scores are zero during the presentation.16:08–22:58 · Matt pushing back 2/10 Fireside Q&A with Matt Turck Host Matt Turck opens the Q&A by asking about company background and probing how they know they are ready for commercial deployment. Audience members follow up with questions on facial expressions, local city variations, and unpredictable actors like children or drunk people. Sam responds collaboratively and clarifies how ambiguity is modeled as a safety signal.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 0% · guest 100%0:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 11% · guest 89%15:00 · Matt 11% · guest 89%18:00 · Matt 1.5% · guest 98.5%18:00 · Matt 1.5% · guest 98.5%21:00 · Matt 0.8% · guest 99.2%21:00 · Matt 0.8% · guest 99.2%
Sharpest disagreement ▶ 3:55 Rejecting general AI approach as Skynet

Sam forcefully rejects the conventional engineering ambition to recreate full human social reasoning via general AI, calling it a 50-year project that risks building Skynet.

 Hardest push from Matt ▶ 17:10 Matt Turck probing commercial deployment readiness

Host Matt Turck pushes back on the complexity of the domain, asking Sam how they can prove their system is ready for commercial deployment.

 Biggest teaching moment ▶ 5:10 Counter-intuitive bag-tightening behavioral cue discovery

Sam educates the audience on how humans subconsciously process subtle micro-behaviors like bag tightening before crossing, a nuance revealed through post-hoc model analysis.

 Matt holds his own ▶ 17:10 Matt Turck drilling into commercial KPIs and deployment roadmap

Host Matt Turck demonstrates domain awareness by pressing the guest on operational readiness and customer performance metrics rather than accepting general product claims.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
The Physics-Only AV Limitation 0320 Sam Anthony opens his presentation by challenging the physics-only approach used by current autonomous vehicles. Because this is a monologue presentation, host metrics are zero. Sam highlights Moravec's paradox to explain why human social intuition is hard for machines.
Subtle Cues and Industry Consensus 0420 Sam details subtle physical cues like bag-holding tension that indicate pedestrian intent, noting industry leaders agree this is the main hurdle. He references reports of pedestrians punching timid self-driving cars to underscore brand risk. As a monologue, host scores remain zero.
Perceptive Automata's Methodology and Data Pipeline 0410 Sam explains Perceptive Automata's methodology of running human behavioral science experiments at scale to train machine learning models. He emphasizes answering questions as a virtual crowd vote rather than building general AI. Host scores remain zero for this monologue section.
Real-World Computer Vision Demonstration 0410 Sam walks through real-world visual demonstrations showing intention and awareness models in action. He highlights an edge case of a jaywalker at a cab stand who breaks standard physics and historical rules, but whose intent is clear to human intuition. Host scores are zero during the presentation.
Fireside Q&A with Matt Turck 3412 Host Matt Turck opens the Q&A by asking about company background and probing how they know they are ready for commercial deployment. Audience members follow up with questions on facial expressions, local city variations, and unpredictable actors like children or drunk people. Sam responds collaboratively and clarifies how ambiguity is modeled as a safety signal.

Statements from this episode (5)

Insight
Sam Anthony: Physics-only autonomous vehicles struggle to predict human intent
“So that sort of obvious inference is something that we as humans do effortlessly, and that if you just have a box, if you have an autonomous vehicle that's just, you know, rendering everything as moving physics objects in the world, Is difficult verging on imp…”
Sam Anthony Nov 13, 2019 ▶ 2:18
Opinion
Sam Anthony: AV developers undervalue human intent inference in driving
“I think that very effortlessness has led to an undervaluing of the difficulty of the situation by people building autonomous vehicles.”
Sam Anthony Nov 13, 2019 ▶ 3:21
Opinion
Anthony: Autonomous driving cannot be solved with sensors and LiDAR alone
“You can't solve it with sensors. You can't solve it with compute power. You can't solve it with, ah, ah, LiDAR. You can't solve it by knowing exactly where everything, everything is.”
Sam Anthony Nov 13, 2019 ▶ 7:44
Assertion Contradicted
Anthony: Pedestrians physically punched Cruise autonomous cars in 2017
“So, in particular I think it was in 2017, the incident reports from crews, there were two or three times where somebody punched the car.”
Sam Anthony Nov 13, 2019 ▶ 8:47
Insight
Anthony: Explicitly modeling human judgment misses most real-world behavioral features
“If you tried to model it explicitly, if I sat here, and you know, I've been doing this for years, if I sat here and wrote down a list of all of the things that matter for these judgments, I would miss the majority of the features that people use in an actual s…”
Sam Anthony Nov 13, 2019 ▶ 19:26
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more