Chollet: Intelligence should be defined as skill acquisition efficiency
Francois Chollet · Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop · Apr 3, 2025 · at 16:06
François Chollet, creator of Keras and the ARC-AGI benchmark and co-founder of NDEA, discusses how true general intelligence should be defined and measured on The MAD Podcast.
0:00 / 0:16exact quote · 16.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And yeah, so I, to summarize that, you know, I see intelligence as skill acquisition efficiency. So it's not the fact that you can acquire skills, it's how efficiently You can do it. That's a measure of your intelligence.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Francois Chollet
Chollet: 50,000x LLM scaling yielded flat progress on ARC benchmark
“Because between, like, GPT-II and GPT-IV. There's been this 50,000 X scale up of base models that has resulted in, in, in basically a flat curve. On Arc.”
Chollet: GPT-4 lacks fluid intelligence, but OpenAI's o3 model has it
“GPT-IV does not have fluid intelligence, for instance, but O-III does.”
Chollet: Commercial AI models will increasingly adopt test-time search architectures
“Increasingly, you're gonna see commercial models that use test-time search, where instead of just trying to generate one single COT to adapt to the task, they're actually gonna run through this, you know, search.”
Chollet: Latest base LLMs score zero percent on ARC-AGI-2
“Today the latest base alarms, they're doing something like 10% on ARK-I. But on Arc two, they are doing zero percent.”