The Exchanges

Q much it costs to do these things, and they'll do a task and they're like, oh, that costs 20 cents. I can't believe I paid that much. How do you think, going back to, like, the product side too, is like, how much do you think of that being your responsibility to try and make it more efficient versus that's not really what we're trying to do with the tool?

A We really see quad code as, like, the tool that gives you the smartest abilities out of the model. Um, we do care about cost insofar as it's very correlated with latency, and we want to make sure that this tool is extremely snappy to use and extremely thorough in its work. We want to be very intentional about all the tokens that it produces. I think we can do more to, like, communicate the cost with users. Um, currently we're seeing costs around, like, Like, six dollars per day per active user, and so it's, like, it does come out to a bit higher, um, over the course of a month in Cursor, um, but I don't think it's, like, out of band, and that's, like, roughly how we're thinking about it.

AI assessment note: “we do care about cost insofar as it's very correlated with latency”

Q you know, exists at a different part of the stack than, than Cloud Code specifically. Cloud Code as a CLI, like, you could use it in any environment, so it's up to you to compose it together. Should we talk about how, how and when models fail? Because I think that was another hot topic for you. I'll just leave it open. Like, how do you observe Cloud Code failing?

A There's definitely a lot of room for improvement in the models, which I think is very exciting. Most of our research team actually uses quad code day to day, and so it's been a great way for them to be very hands-on and, like, experience the model failures, which makes it a lot easier for us to target these in model training and to actually provide better models, not just for quad code, but for, like, all of our coding customers. I think one of the things about The latest Sonnet three seven is, it's a very persistent model. It's like very, very motivated to accomplish the user's goal, but it sometimes takes the user's goal very literally, and so it doesn't always fulfill what, like, the implied parts of the request are, because it's just so narrowed in on, like, I must, like, get x done. And so we're trying to figure out, okay, how do we give it a bit more common sense? So that it, it knows the line between trying very hard and like, no, the user definitely doesn't want that.

AI assessment note: “it sometimes takes the user's goal very literally, and so it doesn't always fulfill”

Q Just to zoom out, you obviously do not have a separate Cloud Code subscription. I'm curious what the roadmap is. Like, is this just going to be a research review for much longer? Are you going to turn it into an actual product? I know you were talking to a lot of CTOs, MVPs, or is there going to be Cloud Code Enterprise? What's the, what's the vision?

A Yeah, so, um, we have a permanent team on Cloud Code. Uh, we're growing the team. We're really excited to support Cloud Code in the long run. And so, yeah, uh, well, we plan to be around for a while. In terms of subscription itself, it's something that we've talked about. It depends a lot on whether or not most users would prefer that over pay as you go. Um, so far pay as you go has made it really easy for people to Start experiencing the product because there's no upfront commitment. And it also makes a lot more sense with a more autonomous world in which people are scripting cloud code a lot more. But we also hear the concern around, hey, I want more price predictability if this is going to be my go-to tool. So we're very much still in the stages of figuring that out. I think for enterprises, given that cloud code is very much like a productivity multiplier for ICs and most ICs can adopt it directly. We've been just, like, supporting enterprises as they have questions around security and productivity monitoring, and so, yeah, we've found that a lot of folks see the announcement and they want to learn more, and so we've been just engaging in those.

AI assessment note: “we have a permanent team on Cloud Code... In terms of subscription itself”

Q Yeah, yeah, exactly. I don't know what we should. Uh, yeah. How do you make decisions? Like, now that the cost of actually implementing the thing is going down as a PM, how do you decide what is actually worth doing?

A Yeah, we definitely still hold a very high bar for net new features. Most of the fixes were like, hey, this functionality is broken or this, like, there's a weird edge case that we hadn't addressed yet. So it was very much like smoothing out the rough edges as opposed to building something completely net new. For net new features, I think we hold a pretty High bar that it's very intuitive to use. The new user experience is, like, minimal. It's just, like, obvious that it works. We sometimes actually use Cloud Code to prototype instead of using docs. Yeah, so you'll have, like, prototypes that you can play around with, and that often gives us a faster feel for, hey, is this feature ready yet? Or, like, is this the right abstraction? Is this the right interaction pattern? So it gets us faster to feeling Really confident about a feature, but it's, it doesn't circumvent the process of us making sure that the feature definitely fits in, like, the product vision.

AI assessment note: “we definitely still hold a very high bar for net new features”

Q Right. Any other comments from memory? Otherwise we can move on to planning and thinking.

A We've been seeing people play around with memory in quite interesting ways, like having Claude write a logbook of all the actions that it's done, so that over time Claude develops this understanding of what your team does, what you do within your team, what your goals are, how you like to approach work. We would love to figure out what the most generalized version of this is, so that we can share broadly. I think with things like Claude Code, with, like, I think when we're developing Things like cloud code. It's actually less work to implement the feature and a lot of work to tune these features to make sure that they work well for general audiences, like across a broad range of use cases. So there's a lot of interesting stuff with the memory, and we just want to make sure that it works well out of the box before we share it broadly.

AI assessment note: “We've been seeing people play around with memory in quite interesting ways”

Q good for the individual developer. But the people who are responsible for the tech, the entire code base, the engineering decisions, all this is going on. My developers, like I manage like a hundred developers, any of them could be doing any of this at this point. What do I do to manage this? How does my code review process change? How does my change management change? I don't know.

A We've talked to a lot of VPs and CTOs about it. They actually tend to be quite excited because They experiment with the tool, they download it, they ask it a few questions, and, like, quad code, when it gives them sensible answers, they're really excited because they're like, oh, I can understand, like, this nuance in the code base, and sometimes they even ship small features with quad code, and I think through that process of, like, interacting with the tool, um, they build a lot of trust in it, and a lot of folks actually come to us and they ask us, like, how, how can I roll it out more broadly? Um, and then we'll often, like, Have sessions with like VPs of dev prod and talk about these concerns around how do we make sure people are writing high quality code. I think in general, it's still very much up to the individual developer to hold themselves up to a very high standard for the quality of code that they merge. Even if we use quad code to write a lot of our code, it's still up to the individual who merges it to be responsible for like this being well maintained, well documented code that has Like, reasonable abstractions, and so I, I think that's something that will continue to happen, where Cloud Code isn't its own engineer that's like committing code by itself. It's still very much up to the ICs to be responsible for the code that's produced.

AI assessment note: “it's still very much up to the individual developer to hold themselves up”