Sep 17, 2019 · 36m · mad
Rebooting AI // Gary Marcus, Robust AI (FirstMark's Data Driven NYC)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Cognitive scientist Gary Marcus examines the fundamental trust and hype issues surrounding modern artificial intelligence, arguing that deep learning alone is too brittle and shallow for complex real-world tasks. He advocates for a new approach that combines statistical machine learning with cognitive science principles and symbolic reasoning to build truly reliable AI.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 6.5% of the talking time here. How this is scored →
Matt as informed peer 1.2 Guest teaching 4.4 Guest disagreement 3.8 Matt pushing back 1.0
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 29:12 Attacking Geoffrey Hinton's opposition to hybrid AI models
Marcus aggressively rejects Turing Award winner Geoffrey Hinton's comparison of classical AI to gas engines and deep learning to electric cars, attributing Hinton's stance to historical bitterness.
Hardest push from Matt ▶ 20:41 Challenging guest on whether brute force methods matterHost Matt Turck directly challenges Marcus's rejection of brute-force deep learning by pointing out that airplanes achieve superior flight through artificial brute force rather than mimicking birds.
Biggest teaching moment ▶ 21:39 Distinguishing closed domains from open-ended real-world outlier limitsMarcus educates the host on why brute force succeeds in closed games like Go but fails in open-ended driving, contrasting human crash rates with Waymo intervention statistics to show the long-tail problem.
Matt holds his own ▶ 20:00 Host quoting book to reframe public AI panicMatt Turck demonstrates close reading of Marcus's book, using a quote comparing modern AI apocalypse panic to 14th-century plague survivors worrying about traffic accidents.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The AI Hype Problem and Refuting Extreme Automation Claims | 0 | 3 | 4 | 0 | In this solo presentation segment, Marcus critiques public AI hype and refutes Andrew Ng's claim that one-second cognitive tasks are easily automated. He highlights trust issues using real-world examples like Teslas crashing into stopped emergency vehicles. | |
| The Underlying Brittleness of Deep Learning and Perceptual Classification | 0 | 4 | 3 | 0 | Marcus breaks down deep learning as mere perceptual classification rather than true intelligence. He revises Andrew Ng's formula to emphasize data dependency and stable domain requirements. | |
| Adversarial Vulnerabilities, Lack of Compositionality, and Visual Misinterpretations | 0 | 4 | 4 | 0 | Marcus presents visual adversarial examples such as sticker-tricked image recognition models and mocks OpenAI's dramatic launch of GPT-2. He demonstrates how language models produce grammatical yet semantically absurd narrative continuations. | |
| Four Lessons from Cognitive Science for AI: Structure, Common Sense, Exploration, and Inna | 0 | 5 | 3 | 0 | Marcus presents four cognitive science principles needed for true AI, including innateness and common sense. He illustrates common sense failures using the Roomba poopocalypse and contrasts human exploratory learning with brute-force trial and error. | |
| Recap and Rules for AI Safety: Studying Small People | 6 | 6 | 5 | 5 | Host Matt Turck engages Marcus in Q&A, offering devil's advocate pushback using the bird-versus-airplane engineering analogy. Marcus defends his hybrid AI stance while forcefully dismissing Geoffrey Hinton's gas-versus-electric engine comparison. |
Statements from this episode (18)
Marcus challenges Andrew Ng: AI cannot automate every one-second mental task
“I don't think it's true at all, so there are some things that people can do in a second that AI can automate, and there are other things that it can't”
Marcus: Teslas crashed into multiple emergency vehicles due to vision flaws
“It turns out that Teslas have a problem with stopped vehicles, mostly emergency vehicles, on the side of the road. So they have run into, in the last year, three fire trucks, a tow truck a police car, and so forth.”
Marcus: Drivers should not trust Tesla Autopilot because AI remains unready
“So you should not be trusting a Tesla. Elon might get on 60 minutes and take his hands off the wheel, but you should not be doing the same because the technology is not really there yet.”
Marcus: Current AI techniques are too brittle to earn public trust
“Most of why I don't think we should trust AI right now is that the current techniques that people are using are simply too brittle to earn the trust that we're giving them.”
Marcus: Deep learning achievements are limited to perceptual classification
“These successes are examples of one thing. They're all examples of what a cognitive psychologist would call perceptual classification, and that's part of what we do as intelligent human beings, but it's not all that we do.”
Marcus: Deep learning requires vast data, quick tasks, and stable domains
“If a typical person can do a mental task with less than one second of thought, and we can gather an enormous amount of data, of directly relevant data, we have a fighting chance. So long as the test data aren't too terribly different from the training data, an…”
Marcus: Deep learning models lack understanding of spatial relationships and silhouettes
“Deep learning doesn't even really fully understand the relations between parts and wholes. It certainly doesn't understand what a silhouette is, and it gets it wrong.”
Marcus: Deep learning relies primarily on texture, not shape, for recognition
“Deep learning, it mostly cares about texture.”
Marcus: Deep learning systems cannot perform compositionality or combine separate ideas
“Deep learning can't do what we call compositionality. It can't put ideas together.”
Marcus: Deep learning fails at reading comprehension and misses the point
“I think that deep learning doesn't just fail on reading. It completely, entirely misses the point.”
Marcus: Deep learning models are giant correlation machines without causal understanding
“These systems are just giant correlation machines. They have no understanding of causation.”
Marcus: Deep learning cannot perform common sense, planning, analogy, or reasoning
“Perception is what deep learning does, and actually it only does a small part of perception. There's lots of parts of perception where we use our knowledge about the world that it doesn't capture. And then there are all these other things like common sense and…”
Marcus: Robust AI was formed to add common sense to robotics
“And, in fact, the point of the company that we formed is to add common sense, which the Roomba lacks.”
Marcus: Deep learning is a better ladder but cannot reach the moon
“Deep learning is a better ladder for sure, but a better ladder doesn't necessarily get you to the moon.”
Marcus: Building human-level AI requires studying cognitive development in children
“If we want to build machines that are as smart as people, we should start by studying small people.”
Marcus: General AI requires hybrid models because cognition has distinct components
“There are a lot of different components to cognition, and so we have to have a hybrid model. There's just no way that one system is going to do all of these different things”
Marcus predicts deeply neurally inspired AI models are 50 years away
“50 years from now, we will know how to build really deeply neurally inspired models, but right now we have to settle for cognitive science, for psychology, linguistics, fields like that, where we can actually get some insight now before we unravel the brain.”