The Exchanges

Q And you think this is slowing down, right? Like in your, um, blog post, you have a pretty striking sentence where you say, uh, GPUs will no longer Improve meaning fully. We have essentially seen the last generation of significant GPU improvements.

A Yeah. So this has two components. And so one is also sort of a very fundamental thing and it's physical in the sense that, um, I mentioned these two components, uh, memory movement and computation. And so computation can only be useful if you move memory to this sort of local neighborhood where you do this computation. Now, this is a geometric problem. You need to have a large store of information, and then use this large store to move information closer to where you want to do the computation. And we have figured out how to physically do this optimal. We have like a large, slow memory. That's DRAM. Then we move it to a cache. If you look at the geometry, that's how you do it fast. If you have a certain size of computation, this is optimal. If you have a different size of computation, matrix multiplication, then you want to use not a CPU, but more like a GPU, which has higher latency, but more throughput. You can move more data, but more slowly. And, um, yeah, if you look at all of that, you can push around a little bit how you structure everything, like the caches, and how large they are, and how much cores are they shared. But in the end, the fundamental problem Uh, remains the same. You have a geometric problem. You can only fill the space in a certain way. And that means you always have certain access pattern with certain, um, latencies. And the biggest latency is a big blo…

AI assessment note: “Yeah. So this has two components.”

Q What are you excited about for 20, 26 at a reasonably granular level? Like what, what do you think happens? What do you think doesn't happen?

A Yeah. So I think they're both sort of, um, I'm sort of split. I think a lot of things will be very boring and not much innovation. But then we also surprised by a couple of things that we maybe don't see. And I think actually sort of in the frontier, we will be less surprised. I mean, it's no secret that we ran out of pre-training data. And as Dan said, like, these are sort of the muscles. Then you can sort of smooth over and you smooth over with synthetic data. And, um, that's how you build coding agents, um, on lots and lots of different environments to combine the data. Um, we make some progress there, but I think you already see the diminishing returns. I don't think coding agents would be that much better. The user experience will probably improve, but, um, you see it that all these models get almost equally good. Like if I use, um, I had my, uh, config like, uh, GLM 4.7 set up and I used it and I thought I was using Opus 4.5. And then I realized, oh, no, wait, wait, I used a different model because they're quite similar. And so I think we see less progress there. Where I think we see more progress is, um, actually the small models. If you train smaller models on more specialized data, they can do quite well. And the smaller models you get, they're pretty powerful. A hundred billion parameter model you can fit it pretty well, even sort of low-grade data center GPU, like an…

AI assessment note: “Where I think we see more progress is, um, actually the small models.”

Q Practically, how do I do that if I'm, uh, Not a coder, and I think about automating some parts of, of my job. What are some of your recommendations on how to approach that problem?

A Yeah, I mean, the best thing is just sort of being sort of very pragmatic. Just think about things and trying to code them. Particularly if you're not a coder, that's very difficult. And there's sort of this barrier with, say, like, I haven't coded before. Like, I don't know this. And, but if you interact with agents, um, they can just build stuff. And with minimal learning, I mean, they can also explain stuff. With minimal learning, you can get there, execute programs, build websites, particularly if it's visual, you get quick feedback. It's not that difficult anymore. I mean, often I mentioned you need to inspect things, but if you build simple tools for yourself to make your life easier, often you don't need to do that. The agents write good code. If you work in a company and you need to integrate it in a good code base, you probably should review it. But if you build a small program on your own to make your work more productive, that's easy. And just to give me an example that might be random here, I put, for example, a tool that if I have a video where I talk, so I record videos of how I interact with agents, then there are certain phases where I just look at outputs and try to understand things. And there are phases where I talk. So I just built a tool that recognizes the speech, that when I'm saying, it's a timestamp, then it slices the video, so basically I have an enti…

AI assessment note: “if you interact with agents, um, they can just build stuff.”

Q Yeah. And your, and your core, your core thinking is that, um, there is a tension between those ideas And the computational reality, is that, is that a fair way to put it?

A Yeah, there are some, there's a physical component, and then there's an idea component, but there's a similar structure, and this structure is basically diminishing returns. Everything that grows exponential will level off, because if you need resources, the resources will be exhausted. Resources can mean different things, and if you look at the physical aspect, um, it gets just more and more difficult to advance technology. That is the case almost with any field of research or development. Um, things get more and more easily. You need more resources to make further progress, and the progress sort of goes lower and lower. And so, if you look at the physical reality of computational devices, and then also computation itself, it has particular structure. And so, basically computation, useful computation is two things. The first is you need to gather data from one location and aggregate it in a certain location where you then put this new information together to compute a transformation of that information. You basically want to combine, um, sort of known things and compute some new sort of, um, things that you didn't know before. Useful information. Useful information can and needs to be transformed from information that you already know. If you move a lot of information around, but you don't transform it, you can't make new information. If you do a lot of computation on the info…

AI assessment note: “Yeah, there are some, there's a physical component, and then there's an idea component”

Q And on that topic of agents, Tim, you just, uh, wrote, uh, another great blog post called, uh, Use Agents So Be Left Behind, and part of what you talk about is coding agents, uh, versus, uh, agents for the rest of tasks. Where are we in that arc where, um, agents are transitioning from being excellent at code to useful for the rest of our lives?

A That blog post was also, as, as a reaction to what I see is, There are large productivity gains if you use coding agents for all kinds of tasks. And as a professor, you don't code that much. You can actually code more easily, which probably other professors previously would. It's so easy now. But yeah, also for non-coding tasks is super useful. And when I look at, um, the productivity gains that I have, some others smaller, like two or three, sometimes it's like 10 times faster. I do tasks 10 times faster. The quality is not degraded. Sometimes the quality is Um, higher, um, an agent might not be as good as I am, but the agent gets to get tired. Agent doesn't, uh, make, uh, sort of bad mistakes or, yeah, and you see cognitively struggle, like, uh, with, like, complicated information that you put together, similar to Pluto kernels, what Dan mentioned. All of that is working. And, um, I mean, Matt, as you put it, is coding agents and agents for other stuff, but how I would see it is, it's just coding agents. Coding agents are general agents. Coding agents can write programs that solve other problems, and code is so general, if there's a digital problem, you could solve it for code, and coding agents make the thing so easy that now you can solve a variety of problems in a way that you couldn't solve before, and this angle makes you productive. I would say this is the main, the mai…

AI assessment note: “how I would see it is, it's just coding agents. Coding agents are general agents.”

Q How do you pick what you want to automate? Like, how do you, how do I think about automation in my life?

A Yeah, so I also, um, talked about that in a blog post, and it can be, yes, sort of, um, sort of, uh, more intuitive thing, and then a more nuanced thing. I think the more intuitive thing is, like, you just think about What could be useful? And then can it be like something more complex to say, I want an Android app or an iPhone app that does this thing. And you initially might think that's complex, but then you throw a correlation and it works immediately. The world's your, uh, it's like, there's so many things that you can do, and you can be very creative and say like, what I always wanted to have, and it wasn't there. Nobody built this product. Can I feel it now? And I think that mindset gives you useful things that makes you more productive, but it also flexes your muscle, and sometimes it doesn't work, and then you understand, like, okay, add aging cycle or this, or this is what I still need to learn, to make both these kind of things, and I think that is sort of the more intuitive perspective that's very useful, and that quickly get you started on the path where you say, like, Um, I mean, first there's excitement, then it's sort of sober, the sober reality, but then you pick up again, and it's like, you realize, okay, if I do it like this, I get more and more productive day by day. That's sort of the more intuitive part. The more nuanced part is the sort of Part that I lea…

AI assessment note: “you look at how you work, you time each of these steps”