Prediction Open · timeframe Mar 2031
Chollet: AGI Codebase Will Be Under 10,000 Lines on 1980s Compute
“I do believe that, you know, when you create a GI retrospectively, it will turn out that it's a code base that's less than 10,000 lines of code. And that if you had known about it back in the 19 eighties, you could have done a GI back then using the computer r…”
Prediction Not checkable as stated
Chollet: Symbolic Models Will Eventually Replicate and Outperform Deep Learning
“And so everything you're doing with machine learning today, with parametric curves, we should be able to do it. With symbolic models in the future in a way that will be much, much closer to optimality. Much closer to optimality in the sense that you're going t…”
Prediction Open · timeframe Mar 2076
Chollet: AI in 50 Years Will Not Use Today's LLM Stack
“I personally don't think that machine learning or AI in 50 years is still going to be built on this stack.”
Insight
Chollet: Gradient Descent Fails at Reasoning by Defaulting to Pattern Matching
“You could not really get Gradient descent to encode sort of like reasoning style algorithms. It was not because the models could not represent these algorithms. It was because gradient descent could not find them, right? So the problem was that it wasn't about…”
Opinion
Chollet: AI field is obsessed with idea that scale alone yields AGI
“Our field became obsessed with the idea that general intelligence would spontaneously emerge by cramming more and more data into bigger and bigger models.”
Assertion Not checkable as stated
Chollet: All inventive AI systems rely on discrete search
“All known AI systems today that are capable of some kind of invention, some kind of creativity, they rely on discrete search.”
Insight
Chollet: Current LLM Stack Can Fully Automate Any Formally Verifiable Domain
“And I think right now we're in this situation where any problem where the solutions you've proposed can be formally verified, and you can actually trust the reward signal. It's not just some guess made by a model. Any domain like this can be fully automated wi…”
Prediction Not checkable as stated
Chollet: LLM Progress in Non-Verifiable Domains Will Slow or Stall
“Progress of reasoning models and base LLMs on this type of domain is, is, you know, it's going to be very slow because the stack we're using, like the LLM stack is very, very reliant on its trained data. It's basically just operationalizing the trained data. A…”
Insight
Chollet: General Intelligence Is Human-Level Skill Acquisition Efficiency
“General intelligence is human level. Skill acquisition efficiency on the same scope of tasks that humans could potentially learn to do.”
Prediction Open · timeframe Mar 2056
Chollet: Future AI Won't Just Be Layered Harnesses on Base LLMs
“Future AI in a few decades it's not going to be this harness on top of a reasoning model on top of a base LLM.”
Insight
Chollet: Human-Engineered Agent Harnesses Prove AI Is Far From AGI
“I mean, to me, the fact that you need humans to engineer these harnesses is also a sign that we're short of AGI today, because if we had AGI, you know, AGI would just make its own harness. It would not need to be told how to solve a problem.”
Insight
Chollet: RL Benchmarks Like Dota and Atari Test Memorization, Not Intelligence
“If you look at Atari games, for instance, or even Dota, you're training on, on the same environment as what you use for testing. So effectively, you're just trying to memorize the best strategies. You're trying to at training time, explore the full space of po…”
Prediction Not checkable as stated
Chollet: AI built from first principles will be more efficient than human brains
“It's an implementation of fundamental principles, the fundamental principles of intelligence, which, you know, I think we can identify these principles and re-implement intelligence from scratch, from first principles, in a way it will be much more efficient t…”
Opinion
Chollet: Replicating Biologically Plausible Brain Mechanics for AI Is Counter-Productive
“I think it would be counter-productive to just try to, you know, observe it and re-implement it, like and make it biologically plausible.”
Insight
Chollet: Human generalization stems from symbolic, causal program synthesis
“I do believe the human mind does at the highest level something that looks a lot like programs in this, like we're currently building Causal models of our surroundings. Like we are describing our surroundings in our mind as, you know, a set of objects and agen…”
Insight
Chollet: Deep learning guidance is necessary to break combinatorial program search
“You have to break the combinatorial wall, and the way to do it is to add deep learning guidance. It's actually very similar to the principles that analyze something like AlphaGo or AlphaZero.”
Prediction Not checkable as stated
Chollet: Artificial General Intelligence Will Likely Arrive Around ARC-6 or ARC-7
“My timeline to AGI, you know, if you just try to extrapolate from the current rate of progress and the amount of investment that's going into, not just the LLM stack, but also like side ideas, side bets that might work out, like, you know, India, for instance,…”
Insight
Chollet: Scaling genetic algorithms could automate scientific discovery
“If you try to scale up genetic algorithms, I mean, I'm sure you can do incredible things with that. You could, in fact, probably do new science. Because that's based on search, and search is the best fit for automating the scientific method.”
Insight
Chollet: AI Architectures Requiring Human Engineers to Scale Will Fail
“If you're working on something, but the only way to increase the capabilities of the system Is to have human engineers and researchers spend time on it. It will not work because even if the idea is very clever and very elegant and works really well, capabiliti…”
Assertion Supported
Chollet: Scaling LLMs 50,000x Only Lifted ARC Accuracy to 10%
“And from at the time, back in 2019 to now, with a model like GPT 4.5, for instance, there's been a roughly 50,000 X scale up of basal alarms. And we went from zero percent accuracy on that benchmark to roughly 10%, which is not a lot.”
Insight
Chollet: Displaying skill across tasks does not demonstrate intelligence
“Intelligence is a process, and skill is the output of that process. The skill itself is not intelligence, and displaying skill at any number of tasks does not show intelligence.”
Opinion
Chollet: Human exam benchmarks cannot measure progress toward AGI
“And that's the reason why using exam-like benchmarks with AI models is a bad idea. They're not going to tell you how close we are to AI. Because human exams weren't designed to measure intelligence. They were designed to measure task-specific skill and knowled…”
Assertion Supported
Chollet: Base LLMs score 0% and static reasoning scores 1-2% on ARC-2
“Well, if you take Bazel Alums, model Slack, GPT-IV-IV-V, LAMA-IV, it's simple, they get zero percent. There is simply no way to do these tasks simply via memorization. Next, if you look at static reasoning systems, so systems that use a single chain of tasks t…”
Assertion Not checkable as stated
Chollet: Gradient descent requires 3 to 4 orders of magnitude more data than humans
“Gradient descent requires vast amounts of data to distill simple abstractions. Many orders of magnitude more data than what humans need. Roughly three to four orders of magnitude more.”
François Chollet