The Exchanges

Q Since you spend so much time on the tool design, so you have this added tool that can make changes and whatnot. Any learnings from that, that you wish like the AI IDEs would take in? Is there Some special way to like look at files, feed them in.

A I would say the core of that tool is string replace. And so we did a few different experiments with like different ways to specify how to edit a file and string replace. Basically the model has to write out the existing version of the string and then a new version, and that just gets swapped in. We found that to be the most reliable way to do these edits. Other things that we tried were like having the model directly, like write a diff, having the model fully regenerate files. That one is actually the most accurate, but it takes so many tokens. And if you're in a very big file, it's cost prohibitive. There's basically a lot of different ways to sort of represent the same task. And they actually have pretty big differences in terms of like model accuracy. I think either, they have a really good blog where they, uh, they explore some of these different methods for editing files and they post results about them. Um, which I think is interesting, but I think this is like a really good example of the broader idea that like, You need to iterate on tools rather than just a prompt. And I think a lot of people, when they make tools for an LLM, they kind of treat it like they're just writing an API for a computer and it's sort of very minimal. It's sort of just the bare bones of what you'd need. And honestly, like it's so hard for the models to use those. I really, again, I come back to …

AI assessment note: “we did a few different experiments with like different ways to specify how to edit”

Q And from the outside, these labs kind of look like huge organizations that have this like obscure ways to organize. How did you get, you joined Anthropic? Did you already know you were going to work on like SweetBench and some of the stuff you publish or you kind of join and then you figure out where you land? I think people are always here to learn more.

A Yeah. I've been very happy that Anthropic is very bottoms up and sort of very sort of receptive to whatever your interests are. Um, and so I joined sort of being very transparent. Like, Hey, I'm most excited about code generation and AI that can actually go out and sort of touch the world or sort of help people build things. And, you know, those weren't my initial, uh, initial projects. I also came in and said, Hey, I want to do the most valuable possible thing for this company and help Anthropic succeed. And, you know, like, let me find the balance of those. So I was working on lots of things at the beginning, um, you know, function calling tool use, uh, and then sort of, as it became more and more relevant, I was like, Oh, Hey, yeah, like let's, it's time to go work on encoding agents. And sort of started looking at SweetBench as sort of a really good benchmark, uh, for that.

AI assessment note: “those weren't my initial, uh, initial projects. I also came in and said”

Q How do we fix that? Are you supposed to fix it at the model level? Like how do I know what prompt I'm supposed to use?

A Yeah. And I'll say this was a very small effect size. And so I think this is not, I think this isn't like worth obsessing over, but I would say that as people are building systems around agents, I think the more you can separate out the different kinds of work the agent needs to do, the better you can tailor a prompt for that task. And I think that also creates a lot of like For instance, if you were trying to make an agent that could both, you know, solve hard programming tasks and it could just like, you know, write quick test files for something that someone else had already made. The best way to do those two tasks might be very different prompts. I see a lot of people build systems where they first sort of have a classification and then route the problem to two different prompts. Um, and that's sort of a very effective thing because one, it makes the two different prompts Much simpler and smaller. And it means you can have someone work on one of the prompts without any risk of affecting the other tasks. So it creates like a nice separation of concerns.

AI assessment note: “first sort of have a classification and then route the problem to two different prompts”

Q But it's not for your actual computer, right? Like the Docker instance is like runs in the Docker. It's not for...

A Yeah, it runs its own browser. I think, um, I mean, the main reason for that is one is sort of security. You know, we don't want You know, the model can do anything. Uh, so we wanted to give it a sandbox, not, not have people do their own computer, at least sort of for our default experience. We really care about providing a nice sort of making the default safe, I think is the, is the best way for us to do it. And I mean, very quickly people made modifications to let you run it on your own desktop. Uh, that's fine. Someone else can do that, but we don't want that to be the official anthropic thing to run. I would say also like from a product perspective right now, Because this is sort of still in beta. I think a lot of the most useful use cases are like a sandbox is actually what you want. You want something where, Hey, any, it can't mess up anything in here. It only has what I, what I gave it. Also, if it's using your computer, you know, you can't use your computer at the same time. I think you actually like want it to have its own screen. It's like you and a person pair programming, but only on one laptop versus you have two laptops.

AI assessment note: “Yeah, it runs its own browser. I think, um, I mean, the main reason”

Q Yeah, yeah, exactly. I saw there's no test writer tool. Is it because it generates the code and then you're running it against Sweebench anyway, so it doesn't really need to write the test or?

A Yeah, so this is, this is one of the interesting things about Sweebench is that The tests that the model's output is graded on are hidden from it. That's basically so that the model can't cheat by looking at the tests and writing the exact solution. But I'd say typically the model, the first thing it does is it usually writes a little script to reproduce the error. Uh, and again, most sweet bench tasks are like, Hey, here's a bug that I found. I run this and I get this error. So the first thing the model does is try to reproduce that. And so it's kind of in rerunning that script as a mini test. But yeah, sometimes the model will like accidentally introduce a bug that breaks some other tests and it doesn't know about that.

AI assessment note: “the first thing it does is it usually writes a little script to reproduce”

Q Yeah. And what about the hardware? A lot of my friends that work in robotics, one of their Big issues. Like sometimes you just have a servo that fails and then you gotta fail and it takes a bunch of time to like fix that. Is that holding back things or is the software still anyway?

A I think both. I think there's, there's been a lot more progress in the software in the last few years. And I think a lot of the humanoid robot companies now are really trying to build amazing hardware. Hardware is just so hard. Um, it's something where classic, you know, you build your first robot and it works, you know, great. Then you build 10 of them. Five of them work, three of them work half the time, two of them don't work, and you built them all the same, and you don't know why. And it's just like, the real world has, like, this level of detail and differences that software doesn't have. Like, imagine if every for loop you wrote, some of them just didn't work. Some of them were slower than others. Like, how do you deal with that? Like, imagine if every binary that you shipped to a customer, each of those for loops was a little bit differently, was a little different. It becomes just so hard to scale and sort of maintain quality. Uh, of these things. And I think that's like, that's what makes hardware really hard. It's not building one of something, but repeatedly building something and making it work reliably. Where again, like you'll, you'll buy a batch of a hundred motors and each of those motors will behave a little bit differently to the same input command.

AI assessment note: “I think both. I think there's, there's been a lot more progress”

Q What was it like when you, when you tried it for the first time? Was it, was it obvious that Claude had reached that stage where you could do computer use?

A It was somewhat of a surprise to me. Like, I think I actually, I had been on vacation and I came back and everyone's like, computer use works. Um, and so it was kind of this very exciting moment. I mean, after the first, just like, you know, go to Google, I think I tried to have it play Minecraft or something and it actually like installed and like opened Minecraft. I was like, wow, this is pretty cool. So I was like, wow, yeah, this thing can actually use a computer. And certainly it is still beta. You know, there's certain things that it's, it's not very good at yet, but I'm, I'm really excited. I think most broadly, not just for like new things that weren't possible before, but as a much lower friction way to implement tool use. One anecdote from my days at Cobalt Robotics. We wanted our robots to be able to ride elevators, to go between floors and fully cover a building. The first way that we did this was doing API integrations with the elevator companies. And some of them actually had APIs. We could send a request and it would move the elevator. Each new company we did took like six months to do, because they were, they were very slow. They didn't really care.

AI assessment note: “It was somewhat of a surprise to me. Like, I think I actually, I had been on vacation”

Q Any other details that are like, Maybe underappreciate it. I know, for example, you had the absolute paths versus relative. Any other fun nuggets?

A Yeah, no, I think that's a good, uh, sort of anecdote to mention about iterating on tools. Like I said, you know, spend time prompt engineering your tools and don't just write the prompt, but like write the prompt or write the tool and then actually give it to the model and like read a bunch of transcripts about how the model tries to use the tool. And I think you will find, like by doing that, you will find Areas where the model misunderstands a tool or makes mistakes, and then basically change the tool to make it foolproof. There's this Japanese term, pokeyoke, about, like, making tools mistake-proof. You know, the classic idea is you have, like, you can have, like, a plug that can fit either way, and that's dangerous, or you can make it asymmetric so that, like, it can't fit this way, it has to go like this, and, like, that's a better tool because you can't use it the wrong way. So for this example of like, uh, absolute paths, one of the things that we saw while testing these tools is, oh, if the model has like, you know, done CD and moved to a different directory, it would often get confused when trying to use the tool because it's like now in a different directory. And so the paths aren't lining up. So we said, oh, look, let's just force the tool to always require an absolute path. And then, you know, that's easy for the model to understand. It knows sort of where it is. I…

AI assessment note: “There's this Japanese term, pokeyoke, about, like, making tools mistake-proof.”

Q Is there a correlation of length of time With like human estimated time, you know what I mean? Or do we have sort of more of ex-paradox type situations where it's something super easy for a model, um, but hard for a human?

A I actually haven't done, um, like done the stats on that, but I think that'd be really interesting to see if like how many tokens does it take and how is that, is that correlated with difficulty? What is the likelihood of success with difficulty? I think actually a really interesting thing, um, that I saw One of my coworkers who was also working on this, uh, named Simon, he was focusing just specifically on the very hard problems, the ones that are said to take a longer than four hours. And then he ended up sort of creating a much more detailed prompt than I used. And he got a higher score on the most difficult subset of problems, but a lower score overall on the whole benchmark. And the prompt that I made, which is sort of much more simple and bare bones, got a higher score on the overall benchmark, but lower score on the really hard problems. And I think some of that is the, the really detailed prompt made the model sort of overcomplicate a lot of the easy problems. Cause honestly, a lot of the sweet bench problems, they really do just ask for a bandaid and where it's like, Hey, this, you know, this crashes, if this is not, and really all you need to do is put a check if none. And so sometimes like trying to make the model think really deeply, like it'll, it'll think in circles and overcomplicate something, which certainly human engineers are capable of as well. Um, but I thi…

AI assessment note: “I actually haven't done, um, like done the stats on that”