Jan 25, 2023 · 1h 1m · big-technology
Is ChatGPT A Step Toward Human-Level AI? — With Yann LeCun, Meta Chief AI Scientist
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Host Alex Kantrowitz interviews Meta Chief AI Scientist Yann LeCun on why contemporary autoregressive language models like ChatGPT cannot achieve human-level intelligence. LeCun advocates for open-source research and introduces alternative cognitive frameworks, including internal world models and Joint Embedding Predictive Architectures (JEPA).
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Alex holds 17% of the talking time here. How this is scored →
Alex as informed peer 4.2 Guest teaching 6.6 Guest disagreement 3.9 Alex pushing back 3.6
speaking balance: gold is Alex, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 25:19 LeCun blasts Galactica critics and internet knee-jerk panic
LeCun forcefully rejects the backlash against Galactica, labeling claims that it would ruin scientific publishing or poison people with glass as completely ridiculous and stupid.
Hardest push from Alex ▶ 26:56 Kantrowitz presses on why Galactica was pulled if backlash was wrongKantrowitz refuses to let LeCun simply dismiss the public panic, directly confronting him with the fact that Meta still took the system down within three days.
Biggest teaching moment ▶ 12:16 LeCun demonstrates ChatGPT's fundamental lack of physical logicAfter Kantrowitz reads a seemingly articulate ChatGPT answer about dropping a sheet of paper, LeCun immediately proves that the model stated the exact physical inverse of reality.
Alex holds their own ▶ 11:27 Kantrowitz tests LeCun's hypothetical directly against ChatGPT liveKantrowitz independently stress-tests LeCun's theoretical claims in real time by executing the exact paper-dropping prompt into ChatGPT and challenging the guest with the output.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Alex as informed peer | Guest teaching | Guest disagreement | Alex pushing back | Why |
|---|---|---|---|---|---|---|
| The Shift in AI Research Economics and Open Science | 4 | 6 | 3 | 3 | Kantrowitz asks insightful structural questions about the business model and open-source incentives of AI research. LeCun educates the host on the economics of FAIR, Google Brain, and why startup research labs struggle without massive corporate backing. | |
| Limitations of Language and the Physical Reality of Intelligence | 4 | 7 | 4 | 4 | Kantrowitz pushes back on the claim that LLMs lack understanding, prompting LeCun with live scenarios. LeCun reframes human intelligence by illustrating non-linguistic physical knowledge with a paper-holding thought experiment. | |
| ChatGPT Hallucinations and the Mechanics of LLMs | 5 | 8 | 5 | 3 | Kantrowitz reads ChatGPT's plausible-sounding answer live, which LeCun immediately deconstructs as physically backwards. LeCun then delivers a deep breakdown of why autoregressive token generation fails at planning and world modeling. | |
| Corporate Risk and the Open Source Foundations of LLMs | 4 | 6 | 4 | 4 | Kantrowitz questions why major incumbents like Google and Meta held back conversational models. LeCun responds by outlining the asymmetric corporate downside and noting that ChatGPT largely repackaged Google and Meta research. | |
| The Backlash Against Meta's BlenderBot and Galactica Models | 5 | 5 | 6 | 6 | Kantrowitz presses LeCun on Meta pulling Galactica and suggests LeCun overestimates public discernment. LeCun vigorously attacks the online outrage as knee-jerk panic and explains the internal decision made by the research team. | |
| Architectures for Autonomous AI and World Models | 3 | 8 | 2 | 2 | Kantrowitz acts as an active listener while LeCun delivers an extended technical overview of his Autonomous Machine Intelligence architecture. LeCun explains self-supervised learning, denoising autoencoders, and why text-based models cannot ground intelligence. | |
| System 1 vs System 2 Thinking and Joint Embedding Architectures | 4 | 8 | 5 | 4 | When Kantrowitz interjects that DALL-E operates like denoising autoencoders, LeCun promptly corrects him. LeCun then explains Daniel Kahneman's System 1 and System 2 thinking and introduces Joint Embedding Architectures for video perception. | |
| Mid-Show Break and Meta Affiliation Overview | 4 | 4 | 2 | 3 | Following the mid-show reset, Kantrowitz asks about Meta's generative video tools and why they remain unreleased. LeCun explains Meta's strategy around generative multimedia and dataset curation requirements. | |
| Machine Emotion, Nick Cave's Critique, and the Authenticity of Art | 5 | 7 | 4 | 3 | Kantrowitz introduces Nick Cave's essay on suffering and emotional authenticity in art. LeCun provides an unexpected counter-thesis arguing that goal-driven autonomous systems will inevitably develop functional machine emotions. |
Statements from this episode (23)
LeCun: ChatGPT is not a major step toward human-level AI
“The short answer is it's not a, Particularly big step towards you know, more like human, human level intelligence.”
LeCun: 'AGI' is a misnomer because human intelligence is specialized
“I don't like the term AGI, artificial general intelligence, because I think human intelligence is very specialized. So if you want to designate the type of intelligence that we observe in, in humans by general, that's a complete misnomer.”
LeCun: OpenAI has become a contract research house for Microsoft
“They've become sort of a contract research house for Microsoft to some extent.”
LeCun: Only large, profitable corporations can economically sustain industry AI research
“The economic model The only one I know outside of universities and philanthropy is industry research lab inside of a large company that has, that is profitable and is sufficiently well-established in this market that it can think for the long term and invest i…”
LeCun: Startups absolutely cannot do fundamental research due to funding
“You can't, absolutely cannot do research in a startup. You just can't because you just don't have the funds, right?”
LeCun: Unclear whether Google's investment in DeepMind has been worth it
“The economic return after 10 years or so, or nine years that Google has gotten from DeepMind, you know, is not clear. It's worth their investment.”
LeCun: OpenAI's technology is well-engineered but not scientifically innovative
“Their technology is not particularly innovative from the scientific point of view. It's very well engineered. So they put together, you know, large scale scaled up system with trained with very well curated data. I mean, they know what they're doing. But in te…”
LeCun: Current LLMs have no real knowledge of physical reality
“Because the understanding that those, the current systems have of you know, the underlying reality that language expresses is extremely shallow. So those systems have only been trained with text a huge amount of text. So they can regurgitate texts that they've…”
LeCun: Most human knowledge is non-linguistic, not captured in text
“How much of human knowledge is present and described in text? And my answer to this is a tiny portion. Like most of human knowledge is not actually language related. It's completely non-linguistic.”
LeCun: LLMs lack internal world models for physical prediction
“LLMs do not have that. They don't have any internal model of the world that allows them to predict.”
LeCun: AI cannot reach human-level intelligence without world models and planning
“And so until we build systems that have some internal model of the world, it allows them to kind of simulate the world if you want, some way of generating actions on the world to use tools like a calculator or something of that, or interrogating a database or …”
LeCun: Most core ChatGPT techniques originated at Google and Meta
“Most of the technology underlying techniques used in ChatGPT have been invented at Google and Meta.”
LeCun: OpenAI relies on impressive demos to raise money from Microsoft
“And they have to produce impressive demos because that's the economic model. That's how they're going to raise money from Microsoft and others.”
LeCun: Incumbents avoid releasing hallucinating AI due to reputational risk
“Whereas if you are Meta or Google, you could think about like, you know, putting out a system of this type that, you know, is going to spew nonsense. And, you know, because you are a large company, you have a lot to lose by, you know, people kind of making fun…”
LeCun: BlenderBot was unimpressive and boring because it was made too safe
“The reason it was, you know, not that impressive in the end is, was because it was made to be safe, essentially.
[1358] Alex Kantrowitz: Okay.
[1358] Yann LeCun: And if it's too safe, it's boring.”
LeCun: LLMs will not ruin scientific conferences because submitting garbage ruins careers
“And I thought that was a completely ridiculous argument, because the reason why you might want to submit a paper is because you want to prop up your CV, and so you have to put your own name on it. Otherwise, what's the point? And if you put your own name and i…”
LeCun: Meta's Galactica AI demo was pulled by its development team
“Well, so what happened was the team that worked on it, which is within FAIR is called Papers with Code. They were so distraught by the reaction that they just couldn't take it. They said like, you know, we're just going to take it down. Like this was not a hig…”
LeCun: Prediction and planning from forecasted consequences define intelligence
“What characterizes intelligence is the ability to predict, first of all, and then the ability to use those prediction Those predictions as a tool to plan by predicting the consequences of actions you might take. Prediction is the essence of intelligence.”
LeCun: Meta will roll out generative AI across its multimedia services
“You're gonna see things like that popping up on, you know, Meta's services in various various places. Generating images, effects, modifying images, generating video, generating music, sound effects, Three D models. Okay. Which of course is important for the me…”
LeCun: Autonomous AI systems will inevitably develop emotions
“So if you have autonomous AI systems that work by optimizing objectives and they have the ability to predict they are going to have emotions. It's inseparable from autonomous intelligence.”
LeCun: AI will not replace genuine art rooted in human emotion
“Even if machines eventually have emotions, they're going to be very different, different from humans. So it's not going to replace this type of genuine art.”
LeCun: AI models should legally be allowed to emulate artistic styles
“Human artists are
Absolutely authorized to get inspired and straight down copy someone else's style.
that happens absolutely all the time in in art.
And so would it make sense to apply a different rule for, let's call them artificial artists, right?
That, tha…”