Brown: Truncating reasoning model thinking mid-sentence still yields good outputs
“So it seems like artificially truncating the thought is actually like fine. Like the model can, even if like it got cut off mid-sentence with an injected like think token, these are smart enough models that they can kind of finish with the best that they got f…”
Brown: Reasoning effort dropdowns will disappear from chat interfaces
“I think in chat interfaces, it probably won't stick around. Like, I don't think we're always going to have the dropdown of like Oath for many and Oath for many high. That feels silly.”
Brown: Claude 3.7 works for quick projects, not large codebases
“I never really got to the point where I found it was helpful for a thing that was like a large existing code base. But if it's like, hey, I want to like cook something up in a few hours for fun. Pretty good at that. But these become messy and they become hard …”
Brown: Small LLMs default to skipping tool calls without explicit training
“If you set these models up to use tools, They just won't. Like if you say, hey, here's a question. You have access to these tools. Do as many rounds of tool calling as you want, and then submit your answer. They'll just submit their answer because they like ar…”
Group-chat multi-agent systems like CrewAI do not work well
“I think people were, like, excited about multi-agent systems for a while, like, the crew AI sort of thing of, like, oh, I'm gonna put my coder agent, my finance agent in group chat, and, like, a lot of these are just kind of silly. They don't actually work ver…”
Algorithms for practical multi-agent RL do not yet exist
“Multi-agent RL's hard. I did five years of it in grad school. It's like not easy. And to the algorithms don't really even exist for the things you would really want to do.”
No legitimate open-source million-token context models exist at scale
“Scaling to, like, million token contexts is, like, really, really hard. There, I don't think there are real, like, open source replications, open token context scaling, Beyond, like, tiny, like, academic model sizes.”
Current multi-turn RL research ignores discarded thinking tokens, breaking the math
“The existing paper people are writing about multi-turn RL are not actually incorporating this, and it kind of, like, breaks all the math.”
Averaging weights of models trained on separate domains works effectively
“You can have a model trained on code, and a model trained on math, and a model trained on Spanish, and you can literally average the weights, and it works.”
Weight updates across specialized tasks are orthogonal enough to merge asynchronously
“The updates made to model weights are orthogonal enough for specialized tasks that this is actually like totally fine. Things are nice and linear in most cases, things are nice and orthogonal, and you can get away with a lot of async updates to models that are…”
Brown: GPT-4.5 Likely Has Trillions of Parameters Enabling Sparse Connections
“GPT, 4.5 is like ginormous model, trillions of parameters, most likely. And that like, there's more room in the model to have these like little sparse connections materialize as you go through layers of the transformer. And I just haven't seen anything like th…”
Will Brown: AI scaling faces diminishing returns on capital investment
“The quality bump over things that are much smaller is just like the, we're hitting diminishing returns on capital investment is a lot of it. Like they're taking out of the API because like the, they can sell other things with the same GPUs and make more money …”
Will Brown: Effective agentic AI will work well on small models
“That's kind of like my bet is like, people are going to really want to train models to be agents. And I think you can get that to work well with a pretty small model.”
Brown: OpenAI's o3 qualifies as 10-minute AGI
“I'm happy to call O three, 10 minute AGI. And I think like framing AGI in terms of like length of time, it takes a human to do a task is like more reasonable than like a global framing. Like, sure. There's a bar of like drop and replace for a human that we are…”