The Exchanges

Q ties with your own work? So, you know, things like, uh, deep learning guided program synthesis, what does that mean? This, this, uh, what does program synthesis mean? And this, um, you know, how, how, how deep learning helps? Um, and then, you know, other, other, some of the other type of approaches were test time training, and then combining program synthesis with transductive models. What does that all mean?

A Yeah, no, I think ArtPrize last year did a tremendous job at highlighting, you know, what are the current best approaches to create models that actually have fluid intelligence, and they're all test time adaptation methods. Um, so in particular, one category of method that, uh, that, you know, really got big via ArcPrize is this time fine-tuning, or this time training, um, where you're using an LLM that's doing, you know, transduction, meaning that it's looking at the task and trying to directly predict the answer. Uh, that's, you know, you can think of it as opposed to doing induction, which is that you look at the task, And you try to predict a program that will, that will turn the task into the answer. So in one case, you just directly predict the answer. In the, in the other case, you try to predict the process or program that gives you the answer. Um, and so you're, you're looking at these, uh, these transductive models. And at this time, you're gonna, um, generate, uh, uh, Input-output pairs from, from the current archetype that you're trying to solve, uh, specifically for fine-tuning, I'm going to fine-tuning your model, uh, to map the one, one input to the outputs, and then, and then you're going to run that model, uh, on the test input, and I'm going to see what it gives you. Um, so that's one thing. Um, another category of approaches, uh, that, uh, that got big with A…

AI assessment note: “program synthesis is this idea that you have some, The language that you're working”

Q And, and, and to this point, so the, the one of the versions of the, Is it the private version? Correct me if I'm wrong. You have to publish exactly what you do. You have to say limit on the amount of compute. Is that fair?

A Yeah. I mean, the, the, the base idea is that there's one track for self-contained approaches with no internet access that are very efficient. So they have very limited compute budget and authors must open source them at the end of the competition. So this is really designed to incentivize, uh, open sharing, uh, and, and, you know, get, get as many ideas as possible. With a big focus on efficiency. We believe, like, efficiency is not just a good feature to have in your system. It's actually at the heart of, of, of intelligence. Um, and the, the other track is to provide continuous benchmarking of frontier models to be able to track, okay, like, if you look at the, at the best currently available commercial frontier models, things like, you know, right now, for instance, O-one Pro, soon the future is going to be O-three and so on, Gemini three, whatever, uh, How much fluid intelligence do these models actually have? Um, mostly independently from, from the efficiency consideration. We still do have, uh, um, this notion that we want to monitor efficiency, so we're going to be reporting, uh, results on a two D plot. Uh, so we're not just looking at the score as a scalar. We're looking at the score associated with, uh, uh, the cost per task, and it was required to achieve this score. And of course, uh, a model that gets you the same score, But at much lower cost per task is a smarte…

AI assessment note: “they have very limited compute budget and authors must open source them at the end”

Q That sounds like the fundamental concept of, of intuition. So is, is part of the idea Uh, of what you're doing and what you're suggesting is to, uh, combine, uh, brute force again, or deep learning for, for what is good at, and then trying to bring that, uh, concept of intuition, uh, as our path to AGI?

A Yeah, I mean, we, we don't know how humans do it, but clearly intuition is part of the picture. So intuition is this idea that you're going to leverage the experience you have, Um, to make fewer guesses. So that, yeah, there are fewer guesses that you have to check before you find the correct guess. A very crude version of that is what LLMs are doing. Um, With an LLM, when you generate a program conditioned on some task definition, effectively you're using a statistical prior about the shape of program space to generate fewer guesses, right? So most of your guesses are going to be syntactically correct, for instance, which is already a nice probability. Most of them are going to be related to the task in some way, and so on. And so if you're using a statistical prior like this, Uh, maybe you can find the right program with, like, a few, a few, uh, uh, millions of programs, uh, uh, or even less than that, a few, like, tens of thousands, maybe thousands of programs, instead of trying billions and billions, uh, which, which is what you would have to do if you were just doing, like, brute force program enumeration. Um, but the, the more sophisticated your intuition is, the fewer guesses you have to make to find the right answer. And, uh, if you, if you get an extremely sophisticated intuition system like humans, we just see the right answer, um, out of, uh, uh, you know, we, we onl…

AI assessment note: “clearly intuition is part of the picture. So intuition is this idea”

Q Please enjoy this wonderful conversation with Francois and Mike. Francois, the big news is the announcement of AHRQ prize, and the concurrent launch of AHRQ AGI-II, but maybe to start from the top, what is AHRQ AGI as a benchmark, and why did you create it?

A AHRQ is an AI benchmark that tries to measure AI fluid intelligence, as opposed to skill on specific tasks. And that's a very different approach from, uh, basically any, any other AI benchmark. Most benchmarks, they're, they're looking at, uh, the ability to answer specific questions, perform specific tasks, that's gonna be very reliant on knowledge, uh, or, or skills, uh, that can be memorized in advance, uh, of trying to pass the benchmark. And AHRQ is not like this at all. AHRQ is a set of tasks that you cannot prepare for. Uh, so it's not trying to measure what you know, it's trying to measure how well you can adapt to something you've never seen before on the fly. Uh, as, as it turns out, this is actually something that's extremely challenging for AI. Uh, current models, they're really good at, uh, absorbing, uh, uh, mountains of, of knowledge and, and specialized skills, but they're not very good at making sense, uh, of novelty on the fly at adapting. This is something that, uh, AI is, is not good at, and that's what makes ARK really interesting. Um, and one, uh, one thing to note is, you know, last year, uh, there was this big Uh, shift, uh, in, uh, in the AI research world where, uh, the AI research community started to move away from, uh, this idea of, uh, just scaling up pre-training and then using, uh, the pre-trained models, uh, in a, in a static fashion at inferenc…

AI assessment note: “AHRQ is an AI benchmark that tries to measure AI fluid intelligence”

Q What are the particular technical challenges, uh, to program synthesis? So if you think of the LLM approach, uh, has been constrained by data, uh, what is a constraint for program synthesis?

A Well, the big constraint is that, um, the, the way you write your program is not very sophisticated, so the probability that the first program that you try is correct is very, very low. To find the correct program, you're going to have to try many, many different programs. For instance, if you're just doing brute force program search, you're trying programs at random, uh, and so to find the right program, you have to iterate with millions of potential solutions of points in the search space, and The main bottleneck is that this search process takes a very, very long time. It's a very large search space, and to evaluate all these points, uh, you know, which point takes you, uh, some amount of competition to evaluate. You're gonna have to evaluate millions of points. Uh, this is extremely expensive, and of course, it's not how humans do it. Humans probably do something akin to, uh, program search, but they're doing very, very little search. You look at the problem, uh, you're, you're quickly building Uh, a very small set of hypothesis to, to explain what you're saying. Maybe you have two, maybe you have three, um, and, and you test them. So it's, it's form of search. It's form program search. Um, uh, with, uh, with backtracking, like if, if one of your programs doesn't work, you can just, you know, mentally debug it or just discard it and move on to the next hypothesis. Um, but t…

AI assessment note: “The main bottleneck is that this search process takes a very, very long time.”

Q And do you know if they fine-tune on the data set that was provided by?

A That's, that's a great question, and we don't have a, uh, A clear answer there. So they told us that they were using a significant fraction, I think they said something like 75%, of the training tasks to, you know, to adapt the model in some way. So of course that's, to start with, you know, that's entirely legit. Like the training tasks are there to train on them. Uh, so if you're training on the, on train data, you know, that's, that's, that's perfectly, that does not invalidate the score that they're getting on the semi-private test set in particular. Um, but what, what did it mean exactly which we were doing to use these tasks? Um, you know, there are, there are at least two interpretations you can have. One is that, oh, they were just part of the pre-training data for the base model. That doesn't really make any sense, because everything was part of the training data for the base model in particular. That's a model that's definitely trained on GitHub. Well, the, the ARC training data is definitely on GitHub, not, not 75% of it, but like, hundred percent of it. And plus, um, the, the public eval data is also on GitHub, so it would also be trained on that, like, definitely. Um, And on top of that, it will also be trained on, like, you know, many, many different repos on GitHub that, that provide more, uh, generated Arc tasks or that provide solution programs, uh, to specific…

AI assessment note: “we don't have a, uh, A clear answer there.”

Q we'll talk about this in a, in a second, but, uh, LLMs, uh, up until oh three, and we'll talk about this. I've been performing very, performed very badly on, uh, RKGI one. The early numbers seem to show that they are not performing particularly well either on, uh, AGI-II. So, what are the fundamental limitations of the LLM approach, um, you know, solely, uh, solely the brute force approach?

A Yeah, absolutely. And it's important to note that, uh, it's not like recently LLMs starting doing well on Arc. LLMs are still doing extremely poorly on Arc. What's actually going on is that we've seen a shift from systems that were just based LLMs used statically at inference time, uh, to much more sophisticated systems that are no longer LLMs, systems that use LLMs as part of a search loop, but fundamentally they're not, uh, uh, end-to-end planning models anymore. In general, like, Uh, LLM is a very overloaded term, uh, in, in different There's, of course, the base elements, where you're just doing, like, you have a prompt. You're just, you know, doing one generation at the inference time. Then you have the so-called reasoning models that are first going through the step of generating a chain of thoughts to sort of, like, improve on your prompt and try to adapt a bit better to the task at hand. Instead of just trying to one-shot the answer. And then you have models that actually perform test-time search, uh, test-time adaptation. Some of them might be doing things like, uh, test-time training, uh, test-time fine-tuning, which is not, not something you see in commercial models, uh, but this technique can apply. And, uh, increasingly, you're gonna see commercial models that use test-time search, where instead of just trying to generate one single COT to adapt to the task, they'r…

AI assessment note: “the limitations of the base models, uh, are still the same.”

Q I noted, um, in your, uh, write-up that, um, there were certain, uh, capabilities that you were focusing on, uh, including symbolic interpretation, compositional reasoning, and contextual rule application. What, what do those mean?

A Well, um, I mean, we can go, if you'd like, we can go, uh, uh, into which each one of them actually represents, but the high level takeaway is that, uh, the new art tasks, they all require some level of, of deliberate deep thinking. You have to think for a while, um, and, um, the, the, the reasoning chain that you're going to have to come up with, uh, is going to have a few hops. In particular, it might, it might have a few control flow hops, like, you know, something like an, an if statement, for instance. Um, and that's something that you were not seeing a lot in arc one, and it really creates, uh, this, this big, uh, difference in terms of how, how challenging the data set is for LMS in particular.

AI assessment note: “we can go into which each one of them actually represents, but the high level takeaway is”