The Exchanges

Q you know, we're selling enterprise SaaS. So if we sell credit packs and you get discounts, if you're an enterprise and you buy a certain amount of credit packs and things like that. So it also just helped the sales motion, um, work a little bit easier. So that's the answer on the abstraction of credits to dollars. Now, Was the question how we decide how to price it, or?

A Yeah, like, I mean, I think there's, all tokens are not made equal, but we obviously get charged mostly equal. Like, you can ask Codex to create you a dumb tool for, like, I created one for our StarCraft II LAN for people to, like, find the game, uh, but then people create it to build features in, like, billion-dollar companies, but the token price is the same. Yeah. Like, for you, I can ask this to update my favorite recipes doc. And it'll do it, but I could ask it to like respond to an email from an investor. And like the value is like very different, you know, and you could charge more, but you're not necessarily doing it. So I'm curious if there was any discussion.

AI assessment note: “Yeah, like, I mean, I think there's, all tokens are not made equal”

Q whatnot. JetTPT obviously has this open AI. So if you give it a memory thing, Yeah, it might use this. But then you have the issue of, well, if I give it this other custom-made MCP that we built internally, and our processes don't map to anything that OpenAI and Anthropic have seen or trained for, it won't be used, and you won't get good results. And super strange, right?

A Yeah, I wrote this article for the GPT-V release about, um, models self-improving for coding. So I basically asked GPT-V What are tools that will be useful to you to be a better software engineer? It's like, well, you know, give a list of like 10 tools. And I'm like, okay, implement them, wrote all the tools. And then I asked it to do the same task I'd done before, but with those tools. And then it goes through the whole task and I'm like, which of the tools did you use? And it's like, oh, I didn't use any of them. And I'm like, why did you not? It's like, you know, to be honest, I don't really need the tools. I can just do this task, you know? And, and I think that's like a good, Metaphor just for, like, the trend of the models, which is like, hey, they're going to use less and less of this, like, custom-made tools to fix today's issue. I think the things that we can bet on, and I'm curious to hear your thoughts, is like, they're always going to have some sort of, like, test runtime. Like, I don't think there's going to be a world in which the model is not going to run tests and say, I'm sure this is going to work. The other one is there's always going to be some sort of, like, Infrastructure as code to then handle the deployment side. So I think whenever there's going to be some runtime issue, they're going to need to understand where they're running, you know, so I think lik…

AI assessment note: “they're going to use less and less of this, like, custom-made tools”

Q But the reason they can't offer at that price is because of the subsidies, right? Which, which is not, like, the long-term, like, sustainable dynamic.

A I mean, it's interesting because, so, both Anthropic and OpenAI are building their own infra, right? And, like, they're gonna get to a place where they're gonna have idle GPUs that they own, and so they will also be incentivized to have a hundred percent utilization, and so You know, they will subsidize some of it just the same way, you know, if you go on SM compute, SF compute, like you pay a buck 40 for like an H 100 instead of like the two 20 listed price on AWS. Um, so I think it will continue, but again, it depends on whether or not they actually have the five hundred billion, like they were saying, which I think they do, you know, just to be clear, I think Stargate will go online, but once it goes online, then it's like, well.

AI assessment note: “so I think it will continue, but again, it depends on whether or not”

Q It can be verbose. You can just have like JSON line outputs and whatnot. Because the agent can understand it much better than a human can. And I think that's just this, a little preview of more things that we will see where you're like, wait a second, this is not made for human consumption anymore. How can we optimize this for agentic consumption? And then maybe the game changes.

A And there's some things that now we get, for example, in my vitest suite, I have a knock to record HTTP calls. So whenever, especially for inference, like you can't really knock You know, we do a classification, things like that. You just need to see what happens. Um, and then we just save the whole interaction, and then the model can actually see what the API return, like in much detail, and like it can reference it back in the future. So when you add a new feature, it can look at the test and it can see what the API usually returns. And it's like, oh, okay, it's gonna have that key and like the content and things like that. I think there's more of that. To be done. I think there was maybe also a time in which having console logs was like really bad. And I think there's maybe not going to be a console log that is like only funneling to like, not the actual console in the browser, but like some way for like the agent to see all of the details of like everything that is happening. What I haven't figured out is like, how do you instrument that? Because you cannot put a whole bunch of console logs that go somewhere else in the code, because then you're also polluting The context window of the model, right? So you need some other way to do it, but I didn't get the more your login, the more the model can kind of like self iterate, but.

AI assessment note: “in my vitest suite, I have a knock to record HTTP calls.”

Q new OS to some extent. So how do you interact with those new apps? So what is an app even in this new world? So that's what is like really, uh, interesting. And that's why I'm glad to work with Rahul, because the guy is so freaking visionary. And if there's one company to nail it, there's not a lot. And I believe like a superhuman is one of them.

A Yeah. I think the inbox is like the ultimate private data source. I feel like even when I see all these companies that are like, you know, talk to like your AI clone to get advice or like, you know, things like that. I feel like so many times, man, I'm just writing the same thing over and over. Like, you know, how many founders email me asking about help for XYZ task? And like, the answer is almost always the same, you know? And like, there should be a way almost for super humans to like, Be the advisor on my behalf, in a way, is that you should be able to predict what I will respond to this email.

AI assessment note: “there should be a way almost for super humans to like, Be the advisor”

Q the best interaction mode? Is it for us to be something super custom for co-work agents to talk to each other? Or is it, okay, let's just jump straight to the finish line and say, well, we're just going to give this thing, if you use Slack at work, we're just going to give this thing a Slack handle, and that's going to be the way it's like multiplayer capable.

A They communicate with each other. Yeah. Like, you know, as, as a fun project, I build this thing called PIQ, which busy takes any repo and the PI agent, uh, coding agent, it puts it in a VPS. And then there's a public webhook where anybody can submit a coding task. And then there's a dashboard in which you reviewed the task. And then there's PIQ. PI PIQ. Yeah. You basically get all these like tasks. Anybody can submit a task. And to me, it's almost like in the organization of the future, it's like the salespeople are talking to the engineering team that is talking to the marketing team, to the product team, and all these co-workers are going to like queue up decisions for other people to approve in a way. Yeah. You know, and I'm kind of curious what that looks like and like, how do you, how do I give my co-work The ability to both approve tasks without asking me. Yeah. And how to decide which one I need to review. Yeah. You know, because for some of these things, it's like, you know, you want to change the color or something. That's kind of like a branding decision. Or another one is like, hey, your thing is just broken. It's like, this is like how you fix it.

AI assessment note: “I build this thing called PIQ, which busy takes any repo and the PI agent”

Q HR and everything. So now we are, I was able to understand how you reply. Typically, but it's always like, if it's covering only 80% of your use cases and you need to discard 20%, where is like the cost benefit value? Is it annoying to have like 20% where you're like, ah, discard. I want to write it myself. Is it good? Like, what is the limit? 9010? 8020?

A I think it's like AI plus the snippets that you have. I think that's kind of like, like I have snippets for a bunch of things, like vendors. I have this like super long snippet. Thank you so much for reaching out about your company. Sounds like a great product. We're not currently in the, blah, blah, blah, blah, blah. It goes on. And then the response is like, thank you so much for your thoughtful response. And I'm like, great, get it out of the way. But I feel like if you could use that plus AI to do the small kind of like last mile thing, I think that would be enough. You don't really need a GI.

AI assessment note: “use that plus AI to do the small kind of like last mile thing”

Q computer, the Pentium three, whatever, there's a weird op from outer space. And sometimes if you bold text in Word, it actually makes it italic, you know, but that's the situation we're in. Like that's That's the fact. Like, it doesn't always boil the text. I mean, it underlines it if you reach a 150 tokens or a 150,000 tokens or something. How do you teach this to the user?

A Yeah. And you know, we're in the church of context engineering at the Chrome office. Um, and when we had Jeff on the podcast, they talked about the context draw out paper that they did. And they mentioned specifically in coding, for example, showing previous failures was like not helpful at all to the agent. And so I think when you're compacting conversation, there's almost like, you know, if you have a long conversation, it usually means something went wrong along the way and you had to like go back and forth and like a bunch of things that didn't work and you're keeping those in. Um, but I've been trying to figure out what's like, what's that gonna look like? In my mind, it's almost like if you take the idea of linear, which I use and I give to my agents just to get, because then I have a canonic prompt For one issue, because often you have to restart because it's like, it just goes too much wrong down the wrong path.

AI assessment note: “we're in the church of context engineering at the Chrome office.”

Q latent space might not be optimized, right? For, for actual, uh, like to actually, um, retrieve the relevant piece of context that the user wants to ask. So can you shift the embedding points a little bit, right? And how do we do that? Basically, that's really the key question here. So optimizing the embedding model, um, even changing the way you like chunk things, these all shift the embeddings.

A So the retrieval is interesting. I got a bunch of startup pitches that are like, like rag is cool, but like there's a lot of stuff in terms of ranking that could be better. There's a lot of stuff in terms of, um, Uh, sunsetting data. Once it starts to become stale, that could be better. Um, are you gonna move into that part too? So like you have SEC Insights as one of kind of like your demos, and that's like a great example of, hey, I don't want to embed all the historical documents because a lot of them are outdated, and I don't want them to be in the context. What's that problem space like? How much of it are you gonna also help with and versus how much you expect others to take care of?

AI assessment note: “So the retrieval is interesting. I got a bunch of startup pitches that are like”

Q of at least a design system and consistent. So it's not distracting because an idea should be evaluated on some merits. But then I think from there, the actual exploration and making that great, that is a hard design task still. So how do we lower the floor for everyone coming in, but also raise the ceiling, make us the designers can do even more and produce even greater work?

A I think like, I mean, obviously, you know, it's cliche, like everything is changing, right? Uh, but I think there's just like a fundamental shift in both how people perceive software. I think, you know, Sam tweeted, some moment tweeted about the fast fashion era of SaaS, but there's also like a negative connotation to fast fashion, but I feel like in software, it's like, man, if you can get the software that you need at any moment, that's not a cheap thing. That's like an expensive thing that is now be made cheap, and it's like still high value, and I'm curious like how different products are gonna drive that, even though it feels like, hey, I just created this for you very quickly. But like, there was so much that went into that, you know, that it's like maybe sometimes undervalued, you know? So I'm curious, like, if there's something that you think about where, okay, in a way people should come to Figma and do a lot of work, but maybe in a way we can kind of help you from like all the work you've done in the past, kind of like come to the right result much faster in the future.

AI assessment note: “So I'm curious, like, if there's something that you think about”