ARC-AGI, every mention
15 scenes, the whole family · ← back to ARC-AGI
tap a year for its mentions
every year anyone Matt Turck 32Francois Chollet 11Mike Knoop 8Sebastian Raschka 4
Verbatim, from the transcripts: the passages where ARC-AGI comes up
State of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka
- ▶ 6:18 Sebastian Raschka And so they are interesting because they were getting very good performance for their very small size on the arc benchmark. 4 times in the scene
Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop
- ▶ 1:10 Matt Turck In the second part of the conversation, we discuss with Mike and Francois their big recent news, the announcement of ARK AGI-II and ARKPrize, which is often known as an IQ test for machines, which historically has tripped up LLMs from big…
- ▶ 2:00 Matt Turck Francois, the big news is the announcement of AHRQ prize, and the concurrent launch of AHRQ AGI-II, but maybe to start from the top, what is AHRQ AGI as a benchmark, and why did you create it? 8 times in the scene
- ▶ 2:00 Matt Turck Francois, the big news is the announcement of AHRQ prize, and the concurrent launch of AHRQ AGI-II, but maybe to start from the top, what is AHRQ AGI as a benchmark, and why did you create it? 3 times in the scene
- ▶ 8:14 Matt Turck I've been performing very, performed very badly on, uh, RKGI one. 8 times in the scene
- ▶ 8:19 Matt Turck The early numbers seem to show that they are not performing particularly well either on, uh, AGI-II. 2 times in the scene
- ▶ 13:33 Matt Turck As further background on, on all of this, uh, so ARK is a, um, benchmark, uh, that ultimately focuses on the progress towards AGI, uh, which gets us into this whole conversation about, uh, what AGI is and what intelligence is, and that's… 2 times in the scene
- ▶ 16:45 Matt Turck Uh, on ArcG one, uh, for, for a bit, and, uh, I believe shortly after the end of, uh, uh, Arc prize, 22,024, uh, you, you guys got a call from, uh, OpenAI and OpenAI said, Hey, you know, actually our, our, uh, check, we're doing some work… 3 times in the scene
- ▶ 21:40 Francois Chollet Well, the, the ARC training data is definitely on GitHub, not, not 75% of it, but like, hundred percent of it. 4 times in the scene
- ▶ 33:49 Mike Knoop I think I first got introduced, Francois, to your work back in, like, during COVID, uh, on the Lex Friedman podcast and rediscovered it during this period at Zapier, and, um, I, I started just, like, surveying people in the AI landscape,… 4 times in the scene
- ▶ 37:18 Francois Chollet Um, if you didn't look at art, this, this, this was, this was the takeaway you had. 4 times in the scene
- ▶ 44:55 Mike Knoop People bring and apply some of those ideas to the, to the new RKGI two data set this year. 3 times in the scene
- ▶ 46:43 Matt Turck Going into, uh, what is new is AHRQ AGI-II, uh, maybe let's talk about the panel. 5 times in the scene
- ▶ 47:09 Francois Chollet We, we were not trying to, like, uh, select for people that were good at solving puzzles, for instance, that would be very strong at the arc. 3 times in the scene
- ▶ 56:09 Mike Knoop I think that's even like in a small way, what things like RKGI two show is missing.