May 15, 2023 · 1h 6m · news
Yann LeCun: Meta’s New AI Model LLaMA; Why Elon is Wrong about AI; Open-source AI Models | E1014 · 20VC with Harry Stebbings
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this comprehensive interview, Meta's Chief AI Scientist Yann LeCun discusses the future of artificial intelligence, advocating for open-source models, debunking existential doom scenarios, and detailing the architectural shift needed to move past limited large language models toward true human-level intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Harry holds 13.2% of the talking time here. How this is scored →
Harry as informed peer 3.1 Guest teaching 3.5 Guest disagreement 2.7 Harry pushing back 2.5
speaking balance: gold is Harry, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 52:02 Calling Musk's view obscurantism
LeCun emphatically dismisses Elon Musk's premise that AI cannot be controlled after release, calling it completely false, ridiculous, and comparing it to historical obscurantism against the printing press.
Hardest push from Harry ▶ 41:00 Stebbings counters with media industry job cutsStebbings directly refutes LeCun's optimistic economic transition timeline by pointing out that AI is actively reducing headcount in his own media business today.
Biggest teaching moment ▶ 14:40 Debunking the dominance fallacyLeCun re-educates the host on AI safety assumptions by separating intelligence from dominance, drawing on primatology examples of territorial orangutans versus social baboons.
Harry holds his own ▶ 35:01 Framing Google's hesitation as Innovator's DilemmaStebbings demonstrates deep business analysis expertise by reframing Google's failure to release ChatGPT-like products through Christensen's Innovator's Dilemma and core ad revenue cannibalization.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Harry as informed peer | Guest teaching | Guest disagreement | Harry pushing back | Why |
|---|---|---|---|---|---|---|
| Yann LeCun's Early Career and Discovery of AI | 2 | 3 | 1 | 1 | Harry sets up the interview with friendly historical questions about LeCun's early career and AI breakthroughs. LeCun provides detailed backstory on neural net history, backprop, and his early collaboration with Geoff Hinton. | |
| Continuous Evolution vs. Splashy Public Milestones | 2 | 3 | 2 | 1 | Harry asks whether current AI progress is an inflection point or continuous development. LeCun explains that public perception sees sudden jumps due to splashy demos, whereas researchers view it as steady evolution. | |
| The Limitations of Autoregressive LLMs and What Is Missing | 2 | 4 | 2 | 1 | Harry asks about recent surprises and current AI missing links. LeCun explains the limitations of autoregressive LLMs and cites Moravec's paradox regarding non-linguistic physical world knowledge. | |
| Debunking AI Doom and the Dominance Fallacy | 2 | 5 | 5 | 2 | LeCun forcefully rejects AI doom narratives and Jeff Hinton's recent warnings as nonsense. He explains that intelligence does not inherently imply a desire to dominate, contrasting social primates with solitary orangutans. | |
| Designing Safe, Objective-Driven AI and Who Sets the Rules | 2 | 3 | 1 | 2 | Harry asks how objective-driven AI can be constrained and who determines safety rules. LeCun proposes a crowdsourced, Wikipedia-style open vetting process for assistant infrastructure. | |
| Why Open-Source Wins and Meta's AI Strategy | 4 | 3 | 2 | 3 | Harry brings up the leaked Google memo and questions why open source beats proprietary models. LeCun highlights Linux/Apache history and explains Meta's strategy of open infrastructure. | |
| Small Models, LLaMA, and Human Data Efficiency | 3 | 4 | 2 | 2 | Harry questions model size moats and efficiency. LeCun explains that LLaMA demonstrated high performance at smaller scales and contrasts human data efficiency with LLM brute-force training. | |
| AI Ecosystems, Galactica, and the Reputational Risk of Large Tech | 3 | 4 | 4 | 2 | LeCun criticizes AI doomers for attacking Meta's Galactica demo on Twitter. He argues big tech companies face asymmetric reputational risk compared to nimble startups like OpenAI. | |
| The Innovator's Dilemma, AI Job Creation, and Wealth Distribution | 5 | 3 | 2 | 4 | Harry pushes back on technical explanations by citing the Innovator's Dilemma regarding Google's ad model cannibalization. LeCun agrees companies must innovate anyway and addresses macroeconomic job creation. | |
| Creative Careers, the Speed of Transition, and the Psychology of Doom | 5 | 4 | 3 | 5 | Harry challenges LeCun's transition timeline by highlighting immediate job cuts in his own media company. LeCun cites economic literature on adoption lags and explains psychological evolutionary bias toward danger. | |
| Geoffrey Hinton's Exit, Meta's Freedom of Speech, and Algorithmic Moderation | 4 | 3 | 3 | 4 | Harry asks about Geoffrey Hinton's resignation from Google and whether LeCun's Meta role compromises impartiality. LeCun notes his unique executive freedom at Meta and defends AI content moderation. | |
| Debunking Musk's "No Correction" Claim and Quickfire on Global Science Incentives | 4 | 5 | 7 | 4 | Harry quotes Elon Musk's warning that AI cannot be corrected post-release. LeCun strongly rejects Musk's stance as obscurantism, comparing AI pause calls to historical bans on the printing press, then surveys global science incentives. | |
| Corporate AI Talent and Yann's Vision for the Next 10 Years | 3 | 2 | 1 | 2 | Harry asks about competitive talent at rival labs and LeCun's 10-year outlook. LeCun discusses talent departures to startups like Mistral and reaffirms his passion for solving common sense in AI. |
Statements from this episode (34)
LeCun: AI will bring a new renaissance by amplifying human intelligence
“AI is going to bring a new renaissance for humanity, a new form of enlightenment, if you want, because AI is going to amplify everybody's intelligence. It's like every one of us will have a staff of people who are smarter than us and know most things about mos…”
LeCun: Machine learning research halted in late 1960s due to Papert's book
“Much of that literature was in the 19 fifties and sixties, and basically stopped in the late sixties because of a book that they killed it, and Seymour Papad was a co-author of that book.”
LeCun: Neural network research faced a decade of ridicule starting in 1995
“There was indeed about 10 years when not only nobody was interested in neural nets, but people were even making fun of it, you know, talking about it in sort of disparaging terms.”
LeCun: Deep learning pioneers conspired in early 2000s to revive neural nets
“Joshua, Jeff and I decided in the early 2000 to basically start a conspiracy to You know, revive the interests of the community in, in neural nets by making them work.”
LeCun: Transformer scaling with self-supervised learning surprised AI researchers
“The fact that self-supervised learning methods applied to transformer architectures Work amazingly well, and they work, you know, way beyond what we could have expected. The fact that we can do basically train systems to understand language, translate language…”
LeCun: AI progress is continuous evolution, not sudden jumps
“And it looks like kind of jumps when you look at it from far away. But when you're in the field, it's more like a continuous evolution.”
LeCun: Current LLMs do not possess human-level intelligence
“So those systems do not have anywhere close to human level intelligence. Okay. Despite what you might think, we are kind of fooled into thinking it because those systems are very fluent with language, but their ability to think, to understand how the world wor…”
LeCun: Human-level AI will not be achieved using autoregressive LLMs
“Well, so first of all, there is no question that eventually AI systems will understand the world in similar ways that, that humans do perhaps better ways but there will not be autoregressive large language models as a type that we're now talking about.”
LeCun: Autoregressive LLMs will be obsolete within a few years
“So my prediction is that within a few years, nobody in their right mind would use auto-reversive LLMs. They'll go away in favor of something more sophisticated and controllable. They can plan its answer as opposed to just produce one word after the other react…”
LeCun: The belief that smart AI desires dominance is a fantasy
“The second fallacy is that there is this idea somehow that the desire to, and the ability to dominate is linked with intelligence, right? So this is a statement that a lot of people are making, including, you know, my friend Jeff Hinton recently. That somehow,…”
LeCun: Getting AI alignment wrong will not destroy humanity
“It's not like if you get it wrong, it's going to destroy humanity.”
LeCun: AI assistants will replace Google and Wikipedia for digital navigation
“People are probably going to use this almost exclusively in the future for their interaction with the digital world. You know, you're not going to go to Google or Wikipedia. You're just going to talk to your assistant.”
LeCun: Public will reject proprietary AI and demand open-source infrastructure
“They would be so pervasive, so much will ride on, on those systems that I don't think anyone will accept that those assistant being behind Event Horizon in a private company, they will insist that the infrastructure is open.”
LeCun: Open-source projects succeed especially when providing basic infrastructure
“So I think that's why, you know, open source projects succeed, particularly when they concern basic infrastructure.”
LeCun: All of OpenAI and the AI world runs on PyTorch
“ChatGPT was developed on PyTorch. Okay. All OpenAI runs on PyTorch. The entire world, in fact, runs on PyTorch, except Google, because they have their own thing, right?”
LeCun: AI Systems Using Objective-Driven Planning Will Be Even Smaller
“And I think it's going to make even more progress because if we go towards the design of AI systems, perhaps along the lines of what I described with objectives and planning, I think those systems could actually be even smaller.”
LeCun: Autoregressive Large Language Models Do Not Perform Real Planning
“Autoregressive LLMs basically don't do planning, or a very simple form of it.”
LeCun: Open LLMs will become infrastructure and create net new jobs
“The scenario I think will happen, and I'm certainly waiting for is the scenario I described earlier, where you have some sort of open platform for base LLMs. So base LLMs basically would be seen as a basic infrastructure like, you know, TCP, IP, Linux, Apache,…”
LeCun: Proprietary AI models will fall behind open-source models
“So I think the proprietary approaches will actually fall behind.”
LeCun: Meta and Google lagged OpenAI due to risk, not tech
“It's just that they didn't have the pressure to produce new products, completely new products that had a lot of risk attached to them.”
LeCun: AI will create as many jobs as it displaces
“No economist believes we're going to run out of jobs because no economist believes that we're going to run out of problems to solve or requirement for human creativity and human communication and stuff like that. So, you know, this is going to create as many j…”
Stebbings: AI Adoption Is Already Reducing Employment at 20VC
“We use it at the media company, and it's cutting our employment.”
LeCun: New Technologies Take 15–20 Years to Impact Economic Productivity
“When a new technology is introduced, let's say the PC, right, with, you know, a graphical user interface, the mouse, et cetera, right, in the mid-nineties, how long did it take to have a measurable effect on productivity, you know, which is the amount of wealt…”
LeCun: AI-driven mass job loss will take 10 to 15 years
“Is it going to make people lose their job like right away? And no, it's going to take a while. It's going to take 1015 years, you know, possibly more.”
LeCun disagrees with Geoffrey Hinton on AI-driven human extinction risk
“So I understand why, why he might have wanted to leave, but I don't agree with him at all with the whole sort of, you know, probability of human extinction or whatever.”
LeCun: AI is the solution to social network harms, not the cause
“So, for example, there's a narrative, a very, very common narrative that AI is the culprit for a lot of the bad side effects of social networks in the past. And in fact, it's completely backwards. AI is the solution to those problems.”
LeCun: Meta overhauled News Feed algorithm in 2017 to eliminate propaganda
“The main news feed algorithm was completely changed so that, you know, there was no clickbaits anymore. There was no, like, you know, news Outlets that could like push their content. That was propaganda, basically, you know, much more effort to take down false…”
LeCun: AI progress enabled automated hate speech removal across hundreds of languages
“And then what the progress of AI over the last few years basically allowed systems to be deployed to do things like taking, take down hate speech relatively reliably in hundreds of different languages, which was basically impossible to do before.”
LeCun: AI hard takeoff fears are false; no real-world exponential lasts
“So this is predicated on an assumption that is just false, which is the existence of a hard takeoff. Right? So the fact that the minute you turn on a super intelligent AI system is going to take over the world. And it's going to escape your control and it's go…”
LeCun: Switzerland is the only European nation matching US academic jobs
“The only European countries that can rival country that rival the US in terms of the quality of job for an academic or a scientist is Switzerland.”
LeCun: Regulating or slowing down AI research is complete nonsense
“Regulating or slowing down research is complete nonsense. It's just obscurantism.”
LeCun: All Google Transformer Paper Authors Have Left for Startups
“So you look at the original paper from Google about BERT or Transformers, right? The thing that revolutionized NLP, all of them have left. Okay. They're all in startups.”
LeCun: Key Creators of Meta's LLaMA Have Left for Startups
“Some of the people who produced Llama, the open source LLMs from Meta. So the key people have left already. Okay, to do startups.”
LeCun: DeepMind, Meta FAIR, and Cohere are best positioned for AGI
“In my opinion, the outfits that are best positioned for this are Claire from on one side and the new DeepMind now, which is, you know, DeepMind plus Google brain. There are a lot of people there who are interested in that question, and I think they are probabl…”