The Exchanges

Q Can you give some examples of these challenging Taser capabilities that people should explore that it can do now that it couldn't six months ago?

A Yeah. So, okay. One example is the model can now rewrite essentially any code base from one language to a different language. It's, Just sort of crazy. Like it's this work that would have taken just like a very long time as an engineer, and now the model's like quite fast at it. So, so one example of this is, um, Cloud Code is built on the BUN JavaScript Runtime. It's a open source JavaScript Runtime. Um, it's an alternative to Node.js. It's kind of a faster node. BUN was written in Zig. Zig is a systems programming language. It's kind of like C, it's, it's very low level. One of the problems with C, with, uh, with Zig is you have to manually manage memory, and so it's quite easy to run into situations where there's like memory leaks and, you know, other memory management issues. And so one thing that the Bund team was doing is they were having Claude fuzz the code base and try to simulate and trigger memory leaks, and they were doing this for, you know, for a long period of time. They were able to find a lot of memory leaks. It was sort of like a case at a time, and that was kind of the capability of the model at the time was doing this fuzzing. And then at some point, Jared on the team was like, okay, let's just like rewrite it. Maybe the model can do this. And I, I think this is like one of these test problems that he kind of threw at the model with every new model generatio…

AI assessment note: “One example is the model can now rewrite essentially any code base from one language”

Q Let's talk a bit about how then you build this new prompt when there's a new model release, like for everyone in the room, everyone will want to try Opus five and they're going to press delete on their system prompt. How do they go about building, rebuilding the system prompt? How do you set up your environment?

A So you do, you do it kind of piece by piece. So the first step is you delete. The next step is you use it. And you don't want to guess what's the instruction that the model needs because you might not predict it correctly. The thing that you want to do is you want to run it. And if it's like a custom agentic product that you're building, you want to kind of run the products. Uh, you want to see where it fails with the model. You want to see what it does well. If you're using quad code, you want to see where it does well with your code base, or maybe where it stumbles over, you know, the architecture or stumbles over something else. And only when you see it repeatedly stumble on the same thing, That's when you add it back. But you don't want to do it too early. Because remember, like, the model is going to read this instruction every single time you use it. So you really want to make sure that the model needs this instruction. I, I think this is sort of the crazy thing about building on models. It's just so different than all the engineering that I've ever done. Like, in the past, when you built on systems, you built these, like, big, beautiful systems, and you really think about the system design up front. You have, like, a big suite of unit tests. You think about everything, and you know, like a re-architecture is a big project, and sometimes it takes months. I've worked on re…

AI assessment note: “The first step is you delete. The next step is you use it.”

Q How do people get better at that and effectively how do people get better at Prompt engineering. Do people still need to do a lot of prompt engineering, or is that changing as well? Tell us about where this is going.

A Yeah, I remember, like, a year ago, one of the most popular job openings was prompt engineer, um, and then it kind of changed, and then I think it became, like, context engineer. So there's these kind of waves of it. I think these will kind of, like, come and go. I, I think the skill nowadays is less about prompt engineering and more about figuring out how do you give Cloud a hard task that seems a little bit too hard? And then how do you make it possible for Cloud to verify its work along the way? And the verification, I think, is probably the single most important thing that people do not get right, largely. Um, one example of this is people were, uh, you know, we have this desktop app for Cloud. And it's built using Electron. We've made it quite fast, so now it's like a pretty awesome experience. Six months ago, it was like sluggish, and it wasn't very reliable. Now it's pretty awesome, and, you know, it's the thing that most of the team uses. As an experiment, though, I wanted to see, like, what would it feel like if it was native? And so what I did is I, I started a quad tag session, and quad tag is just, you know, it's a, it's a new product we have. It's just quad running in Slack. My first question was, hey, tag, do you have access to a macOS runner on GitHub? And, uh, it said no, and then I, I hooked up a runner, so it was able to start a Mac virtual machine, uh, using,…

AI assessment note: “the skill nowadays is less about prompt engineering and more about figuring out how”

Q And that then created this amazing product just that keeps going. So let's talk about, um, What are some areas and how should future founders here think about on hobbling Claude and fixing this product overhang?

A So there's a couple of things that I will think about. One is you should give the model slightly harder tasks than what you think you can do. I think a really common mistake that I see is people are using quad code, they're using quad, and they, they just give it, like, way overly specific instructions. They're like, I want you to do this, but I want you to do it in this way, this way, this way. You must do, like, one, then two, then three, then four. And for modern models, that's actually really not the way to do it. You want to go a little bit higher level. You want to describe the task. You want to describe the guardrails. You want to describe, like, the exit criteria, and then just go with the model cook and come back in a little bit. And I think it'll, it'll surprise you. Like, and again, like, this is just not something that would have worked six months ago, but it does work today.

AI assessment note: “One is you should give the model slightly harder tasks than what you think”

Q Now my last question is, given everything that we talked about, if there's someone here that's studying CS, and you, you learned to program before this era of, uh, AI in genetic coding, what should students still learn the hard way, like the old way?

A So for me, I learned computer science practically. I learned it by teaching myself to code in order to solve problems. Whenever I was doing this, I was doing it to solve a particular problem that I had. So I actually first learned to code on a TI-Eighty-three calculators. Um, this was back in middle school, and, um, I ended up actually writing a guide on the internet for programming TI-Eighty-three calculators. It's still up on the internet somewhere. Um, and it was, uh, it was basic. That, that was my first language. And I, I learned how to program on the calculator, so I could just, like, get better at my math tests by, uh, by cheating on the test. So it was about something practical, you know, like to me as a middle schooler, that was kind of like the most practical thing I could think of. And I ended up getting good grades, and then I got this little serial cable to give the, you know, the programs to my classmates, and they got really good grades. And then the math got a little bit harder. Um, it wasn't something that I could solve in basic anymore, so I kind of went from this, like, you know, like, maybe algebra solver that was written in basic, and I had to solve harder problems. And, um, you know, like, once we got into calculus, I had to run assemblies so that I could write A better solver so I could cheat better on the test now that it was calculus. And so, for me, pr…

AI assessment note: “learn not just the computer science... but learn how to apply it”

Q I guess next conclusion from this, which you have mentioned in the past that basically coding is solved, right? You have mentioned this. Um, I'm curious now that effectively everyone can write software, what separates the exceptional builders from the rest? What, what are the qualities now that everyone can ship code?

A I would give like one caveat. So coding is solved for the kind of coding that I do. It's not solved for everyone. You know, there's still code bases that are like super deep systems code bases where quad still struggles. There's distributed systems where quad still struggles. There's really kind of in the weeds UI verification, like something is off by pixel or something, but it's still not perfect at this. Like Opus five was a big leap in vision and computer use, but it's still not perfect. Um, but I, I'm actually curious for people here, maybe raise your hand if a hundred percent of your code is written using agents. You don't write any code by hand anymore. It's pretty good. Okay. How about more than 50%? Slightly less hands. Maybe about the same.

AI assessment note: “I would give like one caveat. So coding is solved for the kind of coding”