Opinion
Knoop: AGI progress has stalled over the last five years
“You know, my belief is that AGI progress has really stalled out over the last four or five years.”
Opinion
Knoop: ARC-AGI is the only true AGI evaluation that exists
“Arc AGI to best of my knowledge is the only true. AGI eval that actually exists in the world and measures a actually good definition, correct definition of what AGI is, which we can talk about.”
Opinion
Knoop: LLMs are high-dimensional memorization, not general intelligence
“Effectively what large language models do today is they are high dimensional memorization systems, right? They are trained on lots of training data. They're able to find and generalize patterns off of the training data that they're trained on and then apply th…”
Prediction Not checkable as stated
Knoop: Purely scaling language models will not achieve AGI
“Scaling language models purely will not get there.”
Insight
Knoop: Legislating AI research from theoretical capability predictions is dangerous
“I think it's incredibly dangerous to try and make predictions about future capabilities, about where the technology will go and make rules, legislations, laws.
Like prohibiting or enforcing or requiring certain research directions through a theoretical lens.
I…”
Prediction Open · timeframe Jun 2029
Knoop: ARC solution will likely need under 10k code lines, not massive LLMs
“It's quite likely actually that the solution it can be like written in like 10,000 lines of code or less. And it's not gonna require these like, you know, gigantic You know, two hundred billion large parameter models in order to solve it.”
Assertion Supported
Knoop: Language models objectively cannot beat the ARC benchmark
“Just sort of objectively, language models do not work to beat Arc. And people have tried.”
Prediction Not checkable as stated
Knoop: ARC benchmark solution will likely come from an outsider
“I am more confident actually that or I guess I would bet that the solution arc probably comes from an outsider. I think it's probably gonna come from somebody who's sort of not indoctrinated in the current way of thinking about language models and scale.”
Assertion Not checkable as stated
Knoop: Frontier AI labs have stopped publishing technical details
“Frontier AI research is also basically like completely stopped publishing. You know, the GPD four paper had zero technical details. The Gemini paper had zero technical details on the longer context stuff.”
Assertion Supported
Knoop: ARC-AGI benchmark performance only moved from 20% to 34% in four years
“There's an AI lab called lab 42 out of Switzerland that's been running a small annual contest over the last four years to try and beat this eval and state of the art today is. 34% state of the art four years ago when it was first introduced was 20%. So we've m…”
Insight
Knoop: AGI is properly defined as efficient skill acquisition
“Francois definition, which is the one that I think is the right one is this definition that general intelligence is a system that can effectively, efficiently acquire new skill. That's it efficiently acquiring new skill and being able to solve these open-ended…”
Opinion
Knoop: Cheaper compute justifies revisiting unbiased neural architecture search
“And I suspect now over the last four years, we have. We might now have enough compute that's come online at a cheap enough, like kind of cost per flap that some of those old neural architecture search methods we should revisit and relax the search.”
Opinion
Knoop: Narrow AI should be regulated through existing frameworks
“I mean, I think my sort of underlying beliefs on AI are AI should likely get regulated through the existing regulatory frameworks that exist. I don't see a lot of new harm or use cases or damage caused by just the narrow form of AI systems that we have today t…”
Assertion Not checkable as stated
Knoop: Over 50M AI tasks have run on Zapier
“Over fifty million AI tasks have run on the platform to date over the last year and a half or so since we started tracking.”
Opinion
Knoop: In-workflow AI is the dominant adoption pattern today
“Using AI in the middle of a workflow is, is kind of the dominant way people are adopting AI today.”
Mike Knoop