Everything François Chollet said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Chollet: AGI Codebase Will Be Under 10,000 Lines on 1980s Compute
“I do believe that, you know, when you create a GI retrospectively, it will turn out that it's a code base that's less than 10,000 lines of code. And that if you had known about it back in the 19 eighties, you could have done a GI back then using the computer r…”
Chollet: Symbolic Models Will Eventually Replicate and Outperform Deep Learning
“And so everything you're doing with machine learning today, with parametric curves, we should be able to do it. With symbolic models in the future in a way that will be much, much closer to optimality. Much closer to optimality in the sense that you're going t…”
Chollet: AI in 50 Years Will Not Use Today's LLM Stack
“I personally don't think that machine learning or AI in 50 years is still going to be built on this stack.”
Chollet: Gradient Descent Fails at Reasoning by Defaulting to Pattern Matching
“You could not really get Gradient descent to encode sort of like reasoning style algorithms. It was not because the models could not represent these algorithms. It was because gradient descent could not find them, right? So the problem was that it wasn't about…”
Chollet: AI field is obsessed with idea that scale alone yields AGI
“Our field became obsessed with the idea that general intelligence would spontaneously emerge by cramming more and more data into bigger and bigger models.”
Chollet: All inventive AI systems rely on discrete search
“All known AI systems today that are capable of some kind of invention, some kind of creativity, they rely on discrete search.”
Chollet: 50,000x LLM scaling yielded flat progress on ARC benchmark
“Because between, like, GPT-II and GPT-IV. There's been this 50,000 X scale up of base models that has resulted in, in, in basically a flat curve. On Arc.”
Chollet: Current LLM Stack Can Fully Automate Any Formally Verifiable Domain
“And I think right now we're in this situation where any problem where the solutions you've proposed can be formally verified, and you can actually trust the reward signal. It's not just some guess made by a model. Any domain like this can be fully automated wi…”
Chollet: LLM Progress in Non-Verifiable Domains Will Slow or Stall
“Progress of reasoning models and base LLMs on this type of domain is, is, you know, it's going to be very slow because the stack we're using, like the LLM stack is very, very reliant on its trained data. It's basically just operationalizing the trained data. A…”
Chollet: General Intelligence Is Human-Level Skill Acquisition Efficiency
“General intelligence is human level. Skill acquisition efficiency on the same scope of tasks that humans could potentially learn to do.”
Chollet: Future AI Won't Just Be Layered Harnesses on Base LLMs
“Future AI in a few decades it's not going to be this harness on top of a reasoning model on top of a base LLM.”
Chollet: Human-Engineered Agent Harnesses Prove AI Is Far From AGI
“I mean, to me, the fact that you need humans to engineer these harnesses is also a sign that we're short of AGI today, because if we had AGI, you know, AGI would just make its own harness. It would not need to be told how to solve a problem.”
Chollet: RL Benchmarks Like Dota and Atari Test Memorization, Not Intelligence
“If you look at Atari games, for instance, or even Dota, you're training on, on the same environment as what you use for testing. So effectively, you're just trying to memorize the best strategies. You're trying to at training time, explore the full space of po…”
Chollet: AI built from first principles will be more efficient than human brains
“It's an implementation of fundamental principles, the fundamental principles of intelligence, which, you know, I think we can identify these principles and re-implement intelligence from scratch, from first principles, in a way it will be much more efficient t…”
Chollet: Replicating Biologically Plausible Brain Mechanics for AI Is Counter-Productive
“I think it would be counter-productive to just try to, you know, observe it and re-implement it, like and make it biologically plausible.”
Chollet: Human generalization stems from symbolic, causal program synthesis
“I do believe the human mind does at the highest level something that looks a lot like programs in this, like we're currently building Causal models of our surroundings. Like we are describing our surroundings in our mind as, you know, a set of objects and agen…”
Chollet: Deep learning guidance is necessary to break combinatorial program search
“You have to break the combinatorial wall, and the way to do it is to add deep learning guidance. It's actually very similar to the principles that analyze something like AlphaGo or AlphaZero.”
Chollet: Artificial General Intelligence Will Likely Arrive Around ARC-6 or ARC-7
“My timeline to AGI, you know, if you just try to extrapolate from the current rate of progress and the amount of investment that's going into, not just the LLM stack, but also like side ideas, side bets that might work out, like, you know, India, for instance,…”
Chollet: Scaling genetic algorithms could automate scientific discovery
“If you try to scale up genetic algorithms, I mean, I'm sure you can do incredible things with that. You could, in fact, probably do new science. Because that's based on search, and search is the best fit for automating the scientific method.”
Chollet: AI Architectures Requiring Human Engineers to Scale Will Fail
“If you're working on something, but the only way to increase the capabilities of the system Is to have human engineers and researchers spend time on it. It will not work because even if the idea is very clever and very elegant and works really well, capabiliti…”
Chollet: Scaling LLMs 50,000x Only Lifted ARC Accuracy to 10%
“And from at the time, back in 2019 to now, with a model like GPT 4.5, for instance, there's been a roughly 50,000 X scale up of basal alarms. And we went from zero percent accuracy on that benchmark to roughly 10%, which is not a lot.”
Chollet: Displaying skill across tasks does not demonstrate intelligence
“Intelligence is a process, and skill is the output of that process. The skill itself is not intelligence, and displaying skill at any number of tasks does not show intelligence.”
Chollet: Human exam benchmarks cannot measure progress toward AGI
“And that's the reason why using exam-like benchmarks with AI models is a bad idea. They're not going to tell you how close we are to AI. Because human exams weren't designed to measure intelligence. They were designed to measure task-specific skill and knowled…”
Chollet: Base LLMs score 0% and static reasoning scores 1-2% on ARC-2
“Well, if you take Bazel Alums, model Slack, GPT-IV-IV-V, LAMA-IV, it's simple, they get zero percent. There is simply no way to do these tasks simply via memorization. Next, if you look at static reasoning systems, so systems that use a single chain of tasks t…”