The Exchanges

Q Sorry. What, what do you mean by, by like over the next 18 months, there'll be a lot of changes. Like what, what do you foresee? Actually, let's just catch up. Like what's already happened, you know, in like the sort of recent history.

A Yeah. So there's two categories of information that we have, which is what are the models doing and what are the researchers doing? I think the models provide a lot of inspiration in terms of what the, like what's actual frontier is. And that's things like O-three, Gemini, 2.5, Claude. These are a mix of just O-three, I think is the most scaling RL approach. And then Claude and Gemini, 2.5 are very similar with Hybrid reasoning models that you can turn on and off. They, they rolled it out in different ways. So Gemini didn't have hybrid reasoning at launch, but they're given, they brought it in and Cloud had it at launch. One of the most important questions has got to be is, is the O-three path of just a reasoning model or hybrid reasoning models, like, more useful? Do they diverge in their methods for training them? I think the NVIDIA Lama-Nimetron reasoning paper is probably the Most detailed paper on a hybrid reasoning thing. And then deep seek R one is still the canonical recipe on a like reasoning only model. And those are very different approaches. And I don't know if one will win out or not. And then there's just a lot of work on data side and RL methods. I think there's a list, there's a whole list of kind of GRPO complaints that are out there where the math doesn't make sense for certain things.

AI assessment note: “things like O-three, Gemini, 2.5, Claude. These are a mix of just O-three”

Q at it, you had a really good summary of like recent work in multi-tool RL, which was, which had like loop and retool and Toro and all these other things. And I think that this is just like, An area that's super rich for research right now. I just wanted to give you the space to like highlight. What are your favorites? What do you think that people should explore?

A I could share what my moderate ambition, what would be fun research project things is, is you want to create some sort of competitive dynamic or eval, and it has to be so much narrower than what industry is doing. So I, I told you this at lunch, which is like, um, deep research, but only archive papers. So you're like, don't have to do a full index. You have a limited domain. You have to figure out how to measure it or something, or some, I think like act, it's good for academics to work on academic tools. Cause they have, Very high domain expertise. They already know what's going. And just, like, figure out how to make that something that is either very useful to users, if it's gonna be good enough for that, or something you can't hold climb on. And I don't know if this is, like, brainstorming on the fly of, like, take related works out of papers, just look at the text, and break all the links, and make an eval which is filling in hundreds of related works with archive links. Like, that's a fun, deep research style idea. See if you can do it with open models on a set data store with tools. AI too has gone through a lot of discussions with this, which is you, if you're trying to have impact in AI right now, it's as an academic, you have to like level up out of papers to artifacts, which is models, datasets, evals. Datasets and evals are easier for people to have impact on. And …

AI assessment note: “deep research, but only archive papers. So you're like, don't have to do a full index.”

Q Cool. Well, the last question I was saving up was this whole, uh, what, what is Meta doing? Uh, you know, you had a, you actually had a pretty interesting post Back in, when was this? In April, uh, you said LlamaFor, did Meta just push the panic button? I feel like back then, it didn't actually push the panic button, but now they really push the panic button.

A That's fair. I think the panic button at the time was the whole LMSys model not being the model that they released thing, along with a bunch of weirdities about, like, the day of the week they released. But to be a model that claims to be open and then not release the model that is your leading claim is just, like, that is, like, bad execution. Bad execution. Yeah, yeah, yeah. Which is fine. And then the recent stuff, I think, mostly can be boiled down to talent is cheaper than GPUs by a dramatic margin, and At the end of the day, it's like, okay, if we're spending this much, they go to the room and they stare in the mirror and you're like, wait, it might not actually be that ridiculous to spend this money on the top people. It's like, might as well try.

AI assessment note: “I think the panic button at the time was the whole LMSys model”

Q also talked about how the big labs have all this data that they can find long tail things to RL on, and then, kind of, when you put them all together, that fixes it. Do you feel like what we're able to verify is, like, a big bottleneck that, like, the verifications are only done in, kind of, like, these smaller atomic things, and so we cannot really scale that?

A I think my comment was on making, so in this post I was like reflecting mostly on the question of what will agent progress look like relative to modeling progress. So we've had almost three years of modeling progress and we're pretty used to the messaging on that. And it wasn't just about being with the RL on small things, but do any post training to fix a weird behavior. And RL is a very data efficient way if you can get the right signal, but you could also just say like it does this weird Non-verifiable thing. Let's create a hundred or a thousand instructions to include in post-training so that the model does this types of information extraction correctly, or like soft extraction. It's a space that I want to flesh out more with more examples of tasks. It's just, if you've watched Claude Code going, it's like, what is it doing in the background? It's a lot of reading files and even just the compressing context, like that's not I don't think that's really a verifiable thing, but that being messed up, like that's a super crucial skill for long context actions and long longer tasks is just compressing well, and that's going to take some training novelty on how do you, you can effectively modify your training data instead of having all the multi-turn context. You just insert the summary and you want to make the performance stay as well as it's also a cost saving to have shorter co…

AI assessment note: “RL is a very data efficient way... but you could also just say like it does this weird Non-verifiable thing”

Q and should maybe not be generated every time? I feel like, especially in coding, for certain sets of tasks, you want to have similar types of plants. So maybe it's not the right way to ask the model to regenerate a plan every time. There should almost be like plan blueprints as like tools, and then the model fills it in. Like, where do you think the balance should be?

A I think they're reasonable. A plan is obviously an intermediate goal. I just, it seems likely that there's like failures on this kind of planning level. I mean, the same thing goes for these rubrics that are popular, whereas a lot of the technique that is popular for so-called rubric things is you have a prompt and you have a language model generated rubric for that prompt, which is a few specific things that needs to get right. And that's conceptually very similar to making a plan for, for every task. I think whether or not it's Like grading is that you're going to have a different type of abstraction than executing. But I think in what people are seeing is that it's cheaper relative to the effectiveness to just generate it. So like plans are not super long and they probably, they're not that many tokens. So it's probably just kind of like, okay, we do this. Like putting it in my taxonomy might be overselling it where it just needs to be a prompt and you just need to make sure that your model's not too weird at that prompting stage.

AI assessment note: “what people are seeing is that it's cheaper relative to the effectiveness to just generate”

Q Is that proven? Cause I, I don't know. I thought they had their own index.

A Oh yeah. So I don't, I don't have it. I haven't done detailed books. I'm dealing with rumors, but I, I think they'll all do end up doing their own index and it should, it's one of those things that's like Google should have an advantage again, but who knows if they do. I also hinted at this in my post, but it's like Hamish had tried to set this up the same student from RLVR playing with It's like search in an RL model, and it's very easy to get the model to do tools if you prompt it to, but it's very hard to get the like RL model to learn that the tool is useful. And that's why it's to go through these things where it's like 80 failed tool uses and it still gets it or like it stops or it gets it on the 81st.

AI assessment note: “I haven't done detailed books. I'm dealing with rumors”