StarZero All shows Try the podcast clipper →

Francois Chollet

Co-Founder, Ndea · 1 appearance on the record.

computed by AI from the episodes · how this works → · full disclaimer →

engineerscientistfounderauthor@fchollet ↗fchollet.com ↗Wikipedia ↗

François Chollet is the creator of the Keras deep learning library and author of Deep Learning with Python. He created the ARC-AGI benchmark to measure fluid intelligence and co-founded Ndea to pursue artificial general intelligence through program synthesis.

14statements → 9claims → 4claims resolved → 100%fully supported → 4/5average certainty → 2.5/5average debate potential → 4.3/5argument clarity · the sources → 12said about them ↓

4 supported 0 partly supported 0 contradicted 5 not checkable as stated how the 9 claims stand · each chip opens the sources

2 predictions · 7 assertions · 1 opinion · 3 insights · 1 disclosure · every statement was checked. The predictions and assertions are the 9 claims: statements the public record can support or contradict. 4 are resolved, and 5 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.

The record, in short

What the tape says about how Francois argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.

Their most notable supported claim

Assertion Supported
Chollet: 50,000x LLM scaling yielded flat progress on ARC benchmark
“Because between, like, GPT-II and GPT-IV. There's been this 50,000 X scale up of base models that has resulted in, in, in basically a flat curve. On Arc.”
Francois Chollet Apr 3, 2025 ▶ 11:44 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop

Expressed certainty vs assessment result

none yet certainty 1
none yet certainty 2
none yet certainty 3
100% certainty 4
100% certainty 5

weighted support: a fully supported claim counts one, a partly supported claim counts half. Each filled bar is clickable and opens exactly those claims; "none yet" means nothing said at that certainty level has resolved yet

Argument clarity: do they answer the question? how? →

4.3 / 5 directness 4.4 · coherence 4.8 · precision 4.4 · compression 4

answered every one of 8 assessed questions directly

This is a score against a rubric. It is not a rank. Every host question → answer exchange is scored with names hidden on directness, coherence, precision and compression, 1–5 each, on meaning alone: disfluencies are ignored, and only raw unedited episodes count. This is the score that measures thought. Every scored exchange, scores shown → · The rubric and its checks →

How they sound: speaking style how? →

251 words/min while actually speaking · 80.7 um and uh per 1k words

Measured by listening to the audio itself: 5,902 words across 1 episode of raw-level tape, transcribed verbatim with every um and uh kept, each one attributed only where the alignment onto our timed stream is unambiguous. These are measurements of speaking style. We do not rank them: across this corpus, fluency and argument quality are nearly uncorrelated (ρ≈0.2), and smooth talking does not signal clear thinking. How it's measured →

Everything Francois Chollet said on the MAD Podcast that made the record, most notable first. Filter by type, assessment or year in the ledger →

Assertion Supported
Chollet: 50,000x LLM scaling yielded flat progress on ARC benchmark
“Because between, like, GPT-II and GPT-IV. There's been this 50,000 X scale up of base models that has resulted in, in, in basically a flat curve. On Arc.”
Francois Chollet Apr 3, 2025 ▶ 11:44 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Not checkable as stated
Chollet: GPT-4 lacks fluid intelligence, but OpenAI's o3 model has it
“GPT-IV does not have fluid intelligence, for instance, but O-III does.”
Francois Chollet Apr 3, 2025 ▶ 5:00 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Prediction Not checkable as stated
Chollet: Commercial AI models will increasingly adopt test-time search architectures
“Increasingly, you're gonna see commercial models that use test-time search, where instead of just trying to generate one single COT to adapt to the task, they're actually gonna run through this, you know, search.”
Francois Chollet Apr 3, 2025 ▶ 9:47 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Supported
Chollet: Latest base LLMs score zero percent on ARC-AGI-2
“Today the latest base alarms, they're doing something like 10% on ARK-I. But on Arc two, they are doing zero percent.”
Francois Chollet Apr 3, 2025 ▶ 10:50 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Insight
Chollet: Intelligence should be defined as skill acquisition efficiency
“And yeah, so I, to summarize that, you know, I see intelligence as skill acquisition efficiency. So it's not the fact that you can acquire skills, it's how efficiently You can do it. That's a measure of your intelligence.”
Francois Chollet Apr 3, 2025 ▶ 16:06 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Opinion
Chollet: OpenAI o3 is the most advanced test-time adaptation model
“And OSTRI best I can tell is the most advanced the most successful test and adaptation model out there at this time.”
Francois Chollet Apr 3, 2025 ▶ 17:53 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Insight
Chollet: AGI means humans can no longer easily create tasks AI fails
“You have AGI when it's no longer possible to easily come up with tasks that, you know, you and I can do naturally, but no AI system can do.”
Francois Chollet Apr 3, 2025 ▶ 49:37 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Prediction Not checkable as stated
Chollet: NDEA will solve problems previously unsolved by humans in verifiable domains
“The kind of technology we're building on, it's differential advantage that it's going to be capable of solving problems that have never been solved by humans before. That's, you know, that's a very different deal than LLMs, for instance, but effectively only i…”
Francois Chollet Apr 3, 2025 ▶ 56:57 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Not checkable as stated
Chollet: OpenAI o3 cost $10k–$20k per ARC puzzle on maximum compute
“For instance OpenAI O.S. On the highest compute settings that we tried it on for Arc, it was consuming somewhere between, like, 10,000 dollars to 20,000 dollars per task, like, for one little puzzle, which you could normally solve with a base of an API for a f…”
Francois Chollet Apr 3, 2025 ▶ 20:00 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Not checkable as stated
Chollet: OpenAI used about 75% of ARC training tasks to adapt o3
“So they told us that they were using a significant fraction, I think they said something like 75%, of the training tasks to, you know, to adapt the model in some way.”
Francois Chollet Apr 3, 2025 ▶ 20:48 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Supported
Chollet: MindAI dropped out of ARC Prize over open-source requirements
“They ended up dropping out because they did not want to open source their solution. And of course, that meant that they were not eligible for the prize.”
Francois Chollet Apr 3, 2025 ▶ 42:03 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Assertion Supported
Chollet: Average human test score on ARC-AGI-2 is about 60%
“Based on our own testing, an average person in our test sample would score about 60%.”
Francois Chollet Apr 3, 2025 ▶ 46:17 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Disclosure
Chollet: Work has begun on ARC-AGI-3 featuring a brand-new format
“We're already starting to work on version three which you have a brand new format.”
Francois Chollet Apr 3, 2025 ▶ 50:31 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
Insight
Chollet: Program synthesis bottleneck is the costly million-point search space
“The main bottleneck is that this search process takes a very, very long time. It's a very large search space, and to evaluate all these points you know, which point takes you some amount of competition to evaluate. You're gonna have to evaluate millions of poi…”
Francois Chollet Apr 3, 2025 ▶ 27:25 Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop

The other half of the tape: Francois Chollet's own voice is left out of every number here. Other people bring the name up 12 times in 1 episode on the MAD Podcast. every mention, with the transcript →

Who brings them up most Matt Turck 10Mike Knoop 2

Every mention by year

tap a year for its mentions
00811512025episodesmentions
0112025episodes it came up in
007.50.51512025episodesmentions per episode
2025 12 mentions in 1 episode

Appearances (1)

EpisodeDateSpeaking time
Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop Apr 3, 2025 30m
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more