Mar 19, 2025 · 59m · big-technology
Why Can't AI Make Its Own Discoveries? — With Yann LeCun
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
On the Big Technology Podcast, Meta's Chief AI Scientist Yann LeCun explains why autoregressive large language models cannot achieve scientific breakthroughs, details the diminishing returns of text scaling, and proposes Joint Embedding Predictive Architectures (JEPA) alongside open-source research as the path toward human-level intelligence.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Alex holds 22% of the talking time here. How this is scored →
Alex as informed peer 5.3 Guest teaching 6.4 Guest disagreement 4.3 Alex pushing back 3.7
speaking balance: gold is Alex, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 18:27 LeCun forcefully dismisses LLM-based AGI timelines as complete nonsense
LeCun aggressively rejects claims from industry peers about imminent human-level intelligence, calling the concept of a country of genius in a data center complete BS.
Hardest push from Alex ▶ 22:15 Alex challenges AI optimism with enterprise failure ratesAlex refuses to let consumer adoption metrics obscure severe enterprise deployment barriers, citing 5% deep research hallucination risks and low POC production rates.
Biggest teaching moment ▶ 38:20 LeCun calculates sensory data bandwidth to disprove LLM scaling sufficiencyLeCun breaks down optic nerve throughput to mathematically prove that a four-year-old child processes far more rich sensory data than the entire training corpus of modern LLMs.
Alex holds their own ▶ 22:15 Alex demonstrates domain knowledge on enterprise reliability hurdlesAlex cites Benedict Evans' analysis and specific enterprise deployment metrics to challenge the viability of LLM products facing accuracy and cost bottlenecks.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Alex as informed peer | Guest teaching | Guest disagreement | Alex pushing back | Why |
|---|---|---|---|---|---|---|
| Why Large Language Models Cannot Achieve Scientific Breakthroughs | 5 | 6 | 4 | 3 | Alex prompts Yann with Dwarkesh Patel and Thomas Wolf perspectives on whether LLMs can formulate new scientific hypotheses. LeCun firmly reframes LLMs as mere statistical retrieval systems, drawing a distinction using the brain's Broca area versus underlying mental models. | |
| Limitations of LLM Reasoning and Token-Space Architecture | 5 | 7 | 4 | 3 | Alex asks if chain-of-thought allows models to question premises, citing DeepSeek's superficial philosophical reasoning. LeCun explains that real reasoning requires search over continuous abstract spaces rather than bolted-on token-space generation. | |
| Diminishing Returns, Capital Allocation, and System Thinking | 6 | 6 | 4 | 4 | Alex presses on whether LLMs are hitting a training wall given current Capex spending. LeCun details diminishing returns and connects Kahneman's System 1 and System 2 cognitive frameworks to agentic planning limitations. | |
| Scaling Limits, Infrastructure Capex, and Consumer vs. Enterprise AI | 6 | 5 | 7 | 5 | Alex challenges the return on billions invested into LLM-first labs if a paradigm shift is still years away. LeCun aggressively rejects claims of imminent human-level AGI as complete nonsense while defending Meta's Capex as necessary inference infrastructure. | |
| Enterprise Deployment Hurdles and Lessons from Past AI Hype Cycles | 7 | 6 | 3 | 6 | Alex pushes back on consumer optimism by citing enterprise deployment hurdles, high POC failure rates, and Benedict Evans' deep research critique. LeCun agrees with the skepticism, drawing historical parallels to IBM Watson and 1980s expert systems. | |
| AI Winter Risks, Core Intelligence Pillars, and Open Research | 5 | 6 | 5 | 4 | Alex raises concerns about impending AI winter dynamics amid market fluctuations. LeCun outlines the four pillars required for true intelligence and warns investors against backing small startups promising single-bullet AGI solutions. | |
| Intuitive Physics, Sensory Bandwidth, and Generative Video Pitfalls | 6 | 8 | 6 | 5 | Alex suggests text-to-video systems like Sora prove models are acquiring intuitive physics. LeCun bluntly rejects this, using optic nerve data transmission math to show how a toddler acquires intuitive physics far more efficiently than pixel-level generative models. | |
| World Models and Joint Embedding Predictive Architecture (JEPA) | 4 | 8 | 2 | 1 | Alex prompts LeCun to detail Meta's JEPA architecture. LeCun delivers an extended masterclass explaining why generative autoregressive prediction fails on continuous video and how joint embedding representations enable actual world modeling and planning. | |
| Open Source Acceleration, DeepSeek, and Global AI Innovation | 4 | 6 | 4 | 2 | In a rapid wrap-up, Alex asks if open source has surpassed proprietary models following DeepSeek's release. LeCun argues open research iterates faster globally and cites the origins of ResNet in Beijing and LLaMA in Paris to dispel Silicon Valley exclusivity. |
Statements from this episode (23)
LeCun: Large language models are incapable of inventing new things
“We also know that they can hallucinate facts that aren't true but they're really, in their purest form, they are incapable of inventing new things.”
LeCun: Breakthrough scientific AI will not be an LLM
“No, not, not in the current form. I mean, and whatever form of AI would be able to do that will not be LLMs. They might use LLMs as one component.”
LeCun: AI will eventually ask its own questions, taking considerable time
“You know, is, are we going to have eventually AI architectures, AI systems that are capable of not just answering questions that are already there, but solving, giving new solutions to problems that we specify? The answer is yes, eventually. Not with current L…”
LeCun: Chain-of-thought prompting does not produce genuine reasoning in LLMs
“One simple way of
getting NNMs to kind of appear to reason is chain of thought, right?
So you basically tell them to generate more tokens than they really need to in the hope that in the process of generating those tokens, they're going to devote more computa…”
LeCun: Humans and animals reason through mental models, not token space
“A big issue there is, is that when humans or animals reason,
We don't do it in token space.
In other words, when we reason, we don't have to, you know, generate a text that expresses our solution and then generate another one, and then generate another one, an…”
LeCun: LLM scaling hits diminishing returns after exhausting natural text data
“Well, I don't know if I would call it a wall, but it's certainly a diminishing return in the sense that, you know, we've kind of run out of natural text data to train those LLMs where they're already trained with, you know, on the order of you know, 10 to the …”
LeCun: Nobody knows how to build agentic systems beyond regurgitating plans
“Everybody's talking about agentic systems. Nobody has any idea how to build them other than basically regurgitating plans that have, the system has already been trained on.”
LeCun: Scaling up LLMs will not produce human-level AI
“We are not going to get to human level AI by just scaling up LLMs. This is just not going to happen.”
LeCun: Digital superintelligence will not happen within two years
“Whatever you can hear from some of my more adventurous colleagues it's not going to happen within the next two years. There's absolutely no way in hell to, you know, pardon my French. The, you know, the idea that we're going to have, you know, a country of gen…”
LeCun: Most Meta AI infrastructure investment goes toward inference
“Most of the investment, at least for, from the meta side is investment in infrastructure for inference.”
LeCun: Meta AI has 600 million users
“We have three point something billion users six hundred million users of Meta AI.”
LeCun: Meta AI users are less active than ChatGPT users
“Yeah, but they, but it's not used as much as ChatGPT. Right. So the users are not as intense.”
LeCun: AI deployment falters on the last reliability mile, like self-driving cars
“Beyond the impressive demos, actually deploying systems that are reliable is where things tend to falter in, in the use of computers and technologies and particularly AI. This is not you. It's basically you know, why we had super impressive, you know, autonomo…”
LeCun: IBM Watson healthcare project was a complete failure sold for parts
“So, Watson was going to be the thing that, you know, IBM was gonna push and generate tons of revenue by having Watson you know, learn about medicine and then be deployed in every every hospital. And it was basically a complete failure and was sold for parts, r…”
LeCun: Novel AI architectures will reach practical scale in 3 to 5 years
“The question is, you know, when is this going to go from interesting research papers demonstrating a new capability with a new architecture to, you know, architectures at scale that You know, are practical for a lot of applications and can find solutions to ne…”
LeCun: AGI will not be a single event or come from one company
“There's not going to be like a day Before which there is no AGI and after which we have AGI. This is not going to be an event. It's going to be continuous conceptual ideas that as time goes by are going to be made bigger and to scale and going to work better, …”
LeCun: AI cannot reach human-level intelligence by training on text alone
“And it tells you clearly that we're not going to get to human level AI by just training on text. It's just not a rich enough source of information.”
LeCun: Pixel-level video generation will not solve AI world understanding
“Is the right way to approach this problem to train better and better video generation systems? And my answer to this is absolutely no. The problem of understanding the world does not go through the solution to the, to generating video at the pixel level.”
LeCun: Meta's V-JEPA exhibits physical common sense by detecting impossible events
“And you measure the prediction error as you show the video to the system. And when something impossible occurs, the prediction error goes, goes through the roof. And so you can detect if the system has integrated some idea of what, you know, is possible physic…”
LeCun: Open source AI innovates faster than proprietary models can match
“Open source progress is faster and you know, a lot more innovation can take place in the open source world, which the proprietary world may have a hard time catching up with.”
LeCun: Enterprises prototype with proprietary APIs but deploy open-source models
“What we see is for, you know, partners who we talk to they say, well, our clients, when they prototype something, they may use a proprietary API, but when it comes time to actually deploy the product, they actually use Lama or open source, or other open source…”