The Exchanges

Q Have you been surprised by how price sensitive people are around security and code reviews?

A Not in our segment. I would say in the engineering segment, um, And we, we have some engineers use Replit, but the 75% are non-engineers. Engineers are more price sensitive because they have a lot of options. They can use a lot of different products on the, on the market. Now, when, when you're an operations manager using Replit, And you just saved 10,000 dollars on a SaaS software. You've gained, you've saved another, you know, 200,000 dollars on, on headcount. And you're spending an additional thousand dollars to just make sure that the software is more secure. That's like a no brainer. The ROI has been a hundred fold for, for, for companies we work with. On the consumer side, there's more price sensitivity, especially if I'm an entrepreneur just dipping my toes, which is why we reduce the price on our core plan. So I think there's going to be, and you, you, you hinted at that earlier, there's going to be this different models for different use cases or different parts of your journey. If you're just starting out, you don't want to be hit with a thousand dollar bill. You want to be able to play around with 20, 30 dollars before you commit.

AI assessment note: “Not in our segment. I would say in the engineering segment”

Q You have to weigh off. Is it worth that three month advantage? No.

A Right. You, you do. And that's why it makes it so, so freaking hard and why you have to change your mind all the time, right? Because there's a lead up time, but the most important thing is optionality. So in 20, 23, when we were training models, uh, we achieved better coding performance than the, than the state of the art models at the time, GPT 3.5. Right. Um, but then since Sonnet came out of later Opus, uh, The gap is closed by a lot, and they were doing, they were spending tens of billions of dollars, if not hundred billion dollars, making agents work. And that would, that would have been a dumb strategy for us to go and try to compete on that. But now I would say the opportunity opened up again for other reasons. Uh, the, the open source models are getting really good. And we're, you know, we're approaching a certain plateau in how good coding models could get. And so, uh, you can use your data to fine tune a model specifically for your use case. We, I don't know if you saw, but, um, Intercom yesterday talked about their new model that is better at customer support than the frontier models. And so maybe their model is going to be state of the art for three to six months, and maybe six months from now, the models will like zoom back ahead.

AI assessment note: “Right. You, you do. And that's why it makes it so, so freaking hard”

Q How do you think about maintenance in this case? You have ops teams building tools, you have entrepreneurs building tools. You gotta maintain these fuckers. This is hard enough running a business. Are you going to maintain this now?

A This is where Replit shines. And, and, you know, if you talk to Jason or some of our other customers, um, Replit goes way further than any other Vibe coding product on creating more maintainable software. For example, and part of the reason Replit has been slightly more expensive than, than others is that we do a code review for every, for, for every, you know, code change that we make. We spent a lot of tokens on maintenance as much as we spent on creating that software. Replit also has a built-in tester. So if you enable all the power features, whenever you, whenever the agent writes code, goes into a testing phase, spins up a browser, tests everything in the, in the app, Goes into a code review session, reviews that, kicks it back to the coding agent, gives it feedback, you know, the test failed here, the code review is not good. And people enjoy looking at the code review agent because it's kind of a dick. It's like, this looks like AI generated slop. It'll actually say that. And then, and then it goes back. We're also building, um, agents that are sitting in production software. So we already have security agents right now that are sitting in enterprise Deployments and are monitoring activity and, uh, and they're monitoring packages, monitoring for supply chain attacks. And so the thing about AI, any problem AI creates, there's more AI that you can build to solve that prob…

AI assessment note: “Replit goes way further than any other Vibe coding product on creating more maintainable software.”

Q When you look at your model usage today, I've heard you say before, when I was obviously listening to your prior shows, that you have a preference for Anthropic. What does the model usage look like across different providers today?

A Yeah. So Anthropic, the, you know, has been the sort of workhorse for, for over, over a year right, right now. It's, it's like the, the, the core agent loop because it can run for a long time coherently. But the few things have changed. Google's Gemini's models have, um, are the best at price performance, for example. You know, given their price, where do they sit on the period of frontier, right? And so for tasks, for example, like tasks like code search, we might create a sub agent, uh, that is, that is cheaper and has good enough performance. Uh, and we offload that from the main core loop, right? So we now we use, and I wrote this thesis back in 22, I call it the society of models. Now we use models from every provider. Actually, at some point we were sending more tokens to Google than we were sending Anthropic, despite Anthropic being that kind of the core workhorse. And so there's this concept of agent labs, right? We talk about AI labs, but there's agent labs, you know, us cursor, some of these other companies. Our goal is to start with the user problem. What are we trying to fix? What are we trying to build? And walk back to the technology and use whatever model we need to use. In some cases, we build our own models.

AI assessment note: “Anthropic... has been the sort of workhorse... Now we use models from every provider.”

Q Our company is going to be so much smaller in the future. When you look at the capabilities of individual people, do you buy the ideological Silicon Valley? Oh, you know, we're just going to do more and we're going to be so much more capable. What are we actually like? No, we will have dramatically smaller engineering teams.

A I see, I see both. So, so Jason is someone who's, who wants to work with a very lean team is doing better, more than When we had, when he had people on staff. Yesterday I met an entrepreneur at a conference in D.C. that's using Replit. I think he sells board games online. And he says, it's been so transformative on my business. We've saved so much money on, on SaaS. We're selling more, uh, that I decided to use the increased revenue and efficiency to hire more people. He hired eight more people. Uh, there's a customer case study we actually published on our site, Firecrown Media. Um, like a sixty million dollar media company that owns magazines, different properties, and they've been so successful using Repplet for marketing automations and all sorts of things like that, that decided to hire more people that know how to do vibe coding in order to, uh, to, to, to, to, to sell more and do more and build more. So it depends really on the kind of company, but we see companies that were like, we want to get leaner and we want less people. And I think it'll come down to the entrepreneur, the level of ambition, how they want to run their company. For, for, for us at Rap Lit, I think what we want to try to eliminate is, uh, or reduce is a lot of supporting roles. We want builders, right? We want, uh, we want builders and we want salespeople because I think salespeople, it's like people…

AI assessment note: “I see, I see both. So, so Jason is someone who's, who wants to work with a very lean team”

Q Because you value their creativity. Like why is that? Unpack that logic for me.

A Because Their leverage will be, will be huge. John Carmack is one of the best software engineers in the world. He, he made Doom. He worked on VR and now he's working on AI. He's still limited by how fast he types. Right. Which is like a crazy thought, right? Like if he's not, because, because ultimately you have to type the code out, right? If the AI is actually better at typing the code out, and he's actually manipulating, uh, things at a higher level of abstraction, and he's talking to not just one or two, maybe 1000 AIs doing work for him, he's going to have an insanely higher leverage on the world. And so I think The best, most creative people will have a lot of the menial work automated for them. They wouldn't have to type a unit test anymore. The AI would do that, for example. And so a lot of their time is spent just doing more things and Um, you know, think about Elon, right? Elon is like starting companies left and right, and that's probably not the end of it. Like he'll probably start more companies in the future, right? You know, part of the reason is because he's sort of like operating at a much higher level because he's built a set of very competent people around him. Uh, and like they follow him from one company to another, whether it's like on the finance and legal side, Or it's on these sort of like engineering side. A lot of that could be AI and could be automat…

AI assessment note: “Because Their leverage will be, will be huge.”

Q For example, how are you provocative in phrasing? Cause I find everyone's like, we look for a team player who's good at communication and shared interests and alignment. And you're like, Fuck off.

A Yeah. Zuck said, um, said something very, uh, very interesting, uh, about like move fast and break things. I think a journalist was asking him about it. And he said, like one heuristic to know whether you're saying something meaningful is that like reasonable people can disagree with it. And the opposite of it is also reasonable. So like move fast and break things could be like move slow and don't break anything. And that could be like a value at IBM. Right. Uh, you know, or steady or slow, you know, you can phrase it like a little more charitably. And so if something where the opposite of it doesn't make sense, or nobody would actually, it's, it's not a value anyone would hold, they're actually not saying anything. Uh, it cancels each other out. And so to be provocative, um, you need to say something that people will disagree with, and the opposite of is also somewhat reasonable. One of our values, for example, is like seek pain. And so the idea behind seek pain is that there's a lot of painful things in, in building a startup. Talking to customers is actually extremely painful. When your product is not working or there's like some fundamental, you know, lack of product market fit or some issue you're dealing with, you know, founders and entrepreneurs typically don't want to face that. And facing that is painful. At Replit, we've, you know, had some things in the past, uh, tha…

AI assessment note: “One of our values, for example, is like seek pain.”

Q Why, why do you think we're at the new start now? I love Marc Andreessen's statement that there's no such thing as a bad idea, only the bad time. And it's like, timing is everything I've learned in investing. Why is now the time? Because everyone always says now is the time. Why is now the time?

A Well, I like, we don't have to speculate, right? Like we actually can look at the results. Andre Karpofi was the head of AI at Tesla, built a self-driving team there. He now, I think, back at OpenAI. A few months ago, he tweeted that AI is writing 80% of his code today. 80% of his code. Is written by a machine. That is unprecedented. On Replit, we see that 30 or 50 to 50% of code for Ghost Rider or AI, Ghost Rider users is written by the AI. The Ghost Rider users report that tasks sometimes are cut in half in terms of, like, how much time they needed. There was a study actually done on Copilot users, GitHub's Copilot, Uh, that showed that programmers are 55% more productive. So, okay, this is not a 10 X, right? But it is a start of something. We're really at the early innings of this technology. This technology being large language models and transformer based models. And so it's only been, uh, you know,

AI assessment note: “we don't have to speculate, right? Like we actually can look at the results.”

Q You have to weigh off. Is it worth that three month advantage? No.

A Right. You, you do. And that's why it makes it so, so freaking hard and why you have to change your mind all the time, right? Because there's a lead up time, but the most important thing is optionality. So in 20, 23, when we were training models, uh, we achieved better coding performance than the, than the state of the art models at the time, GPT 3.5. Right. Um, but then since Sonnet came out of later Opus, uh, The gap is closed by a lot, and they were doing, they were spending tens of billions of dollars, if not hundred billion dollars, making agents work. And that would, that would have been a dumb strategy for us to go and try to compete on that. But now I would say the opportunity opened up again for other reasons. Uh, the, the open source models are getting really good. And we're, you know, we're approaching a certain plateau in how good coding models could get. And so, uh, you can use your data to fine tune a model specifically for your use case. We, I don't know if you saw, but, um, Intercom yesterday talked about their new model that is better at customer support than the frontier models. And so maybe their model is going to be state of the art for three to six months, and maybe six months from now, the models will like zoom back ahead.

AI assessment note: “Right. You, you do. And that's why it makes it so, so freaking hard”

Q When you look at your model usage today, I've heard you say before, when I was obviously listening to your prior shows, that you have a preference for Anthropic. What does the model usage look like across different providers today?

A Yeah. So Anthropic, the, you know, has been the sort of workhorse for, for over, over a year right, right now. It's, it's like the, the, the core agent loop because it can run for a long time coherently. But the few things have changed. Google's Gemini's models have, um, are the best at price performance, for example. You know, given their price, where do they sit on the period of frontier, right? And so for tasks, for example, like tasks like code search, we might create a sub agent, uh, that is, that is cheaper and has good enough performance. Uh, and we offload that from the main core loop, right? So we now we use, and I wrote this thesis back in 22, I call it the society of models. Now we use models from every provider. Actually, at some point we were sending more tokens to Google than we were sending Anthropic, despite Anthropic being that kind of the core workhorse. And so there's this concept of agent labs, right? We talk about AI labs, but there's agent labs, you know, us cursor, some of these other companies. Our goal is to start with the user problem. What are we trying to fix? What are we trying to build? And walk back to the technology and use whatever model we need to use. In some cases, we build our own models.

AI assessment note: “at some point we were sending more tokens to Google than we were sending Anthropic”

Q How do you think about maintenance in this case? You have ops teams building tools, you have entrepreneurs building tools. You gotta maintain these fuckers. This is hard enough running a business. Are you going to maintain this now?

A This is where Replit shines. And, and, you know, if you talk to Jason or some of our other customers, um, Replit goes way further than any other Vibe coding product on creating more maintainable software. For example, and part of the reason Replit has been slightly more expensive than, than others is that we do a code review for every, for, for every, you know, code change that we make. We spent a lot of tokens on maintenance as much as we spent on creating that software. Replit also has a built-in tester. So if you enable all the power features, whenever you, whenever the agent writes code, goes into a testing phase, spins up a browser, tests everything in the, in the app, Goes into a code review session, reviews that, kicks it back to the coding agent, gives it feedback, you know, the test failed here, the code review is not good. And people enjoy looking at the code review agent because it's kind of a dick. It's like, this looks like AI generated slop. It'll actually say that. And then, and then it goes back. We're also building, um, agents that are sitting in production software. So we already have security agents right now that are sitting in enterprise Deployments and are monitoring activity and, uh, and they're monitoring packages, monitoring for supply chain attacks. And so the thing about AI, any problem AI creates, there's more AI that you can build to solve that prob…

AI assessment note: “Replit goes way further than any other Vibe coding product on creating more maintainable software.”

Q Like a hundred. What do you invest in for yourself that has been a game changer? I hope it's not too personal, but you've lost a lot of weight. You look fantastic, dude. Um, tell me about that.

A You know, I, I losing the weight has actually been, been like Three years. I decide, I always yo-yoed my weight depending on my stress level at the company, and I decided in 22 or three, I'm gonna take three years to get healthy. And I'm, and instead of like sprinting and working out five days a week and then burning out, I'm gonna work out one day a week. And I started working out one day a week, and then my energy levels improved. I lost some weight. Ok, now I'm going to add another day a week. And ok, now I'm going to add another habit. Let's add a walk. Let's add like a bit of a stretching. And it's so slow. It's sort of like, I want to get healthy over the next five years, and that's been a major game changer. So, I'll just like add another habit every once in a while. It's like very simple stuff. Let's add a walk. Let's add a sauna. Uh, once a week. So I really love sauna and, and cold, um, hot and cold. It, it, it's like a reset. It forces you, I'm, I'm someone who's like constantly thinking, and I tried meditation, all that stuff, and it's really hard for me. When I go into a sauna and get burnt, and then go into the, into like ice cold water, you can't think. Your, your mind freezes, and that forces me into a meditative state for the next like hour or so.

AI assessment note: “I decided in 22 or three, I'm gonna take three years to get healthy.”

Q The thing that I always hear is that we're using frontier models to basically set benchmarks. We're seeing where those benchmarks lie, and then we're switching to open source to get as close to them as possible with much more efficiency in terms of cost. Is that the future?

A It depends, right? Like, so cost question, I think is secondary to the performance question, right? Especially in a, in a, in a time when, when it's flush with capital. Uh, I think when you focus on cost is when you reach a certain Sort of, uh, asymptotic, you know, plateau in the S curve, and you don't foresee, um, a massive improvement, specifically in your domain. Like Intercom might say, you know, we, we don't predict that models are going to get that much better on customer support for the foreseeable future. Therefore, you know, we, we can focus on, on building our own Or there's like a data flywheel that we can, we can get, or there's a cost advantage that we get. But if you focus on cost at the expense of performance, you're going to lose. Uh, and so, you know, it's similar to any era in tech, right? Like it's cloud, it's mobile, it's a SAS, whatever it is, there are moments of time where, where the goal is growth and performance and being at the edge. And then when things kind of rationalize or Uber lift, when things rationalize, you, you kind of focused on your gross margins.

AI assessment note: “cost question, I think is secondary to the performance question”

Q Enterprises will kind of try things in, in certain respects. You know, I was chatting to Jason Lemkin before, and he said, you're number one ICP's product teams. As product teams kind of code or vibe more, what do they do okay in two to three years? How do they think about those functions?

A There's a, there's a big question about how product teams will look in the future. If I were to make a prediction, I would say that we'll still have engineers inside the organizations. Those engineers are responsible for more infrastructure, AI, ML, uh, embedded systems, you know, more, more, you know, low level engineering. And then you have product organizations and product organizations will have people that are like tilt a little more technical and have people tell it a little more design, have people tell it a little more product. But you're not really calling them anything different. They're like a, they're product builders, and their responsibility is to figure out what to build next. I agree with Jason that a lot of the ICP right now is product, but one thing I'm really excited about is operations teams, and they're kind of underserved. Like, operations teams are sitting at the nexus of a lot of data flow. And typically they'll buy a lot of SaaS software. They're typically not happy with it because there's all these SaaS softwares like styling the data. They try a lot of automation software that doesn't work very well. They have a lot of Excel sheets, a lot of manual work. And so, you know, we see a lot of our customers are building, you know, quote, uh, you know, quote configurators for their sales team, automating their deal desk, um, you know, automatic support, uh, …

AI assessment note: “If I were to make a prediction, I would say that we'll still have engineers”

Q Are IDs dead? Will we have IDs in two years?

A I think, I think for all intents and purposes, IDEs are dead. Uh, I think the, the, you know, the limp along because again, some engineers just love that, that control, but there's no future in them and that there's no one's going to be asking for like the latest feature of like IntelliSense or like what were IDEs? Like IDEs, like one part where like the code intelligence, we call it intelligence. It wasn't very intelligent. And so all of that is, is, is irrelevant. Like the autocomplete, the click to symbol, all of that stuff is irrelevant. So in that sense, ideas are dead because AI has like eaten all of that. But in terms of like, You know, people who want to see the code, I think there's still a population of users that want that. I think there's still, you know, engineers that are working with software that they want to actually verify that it works. For example, life or death software. If I'm writing, like, mission-critical software for self-driving cars or, or a NASA or SpaceX mission, I think there's always going to be a need for some kind of IDE.

AI assessment note: “I think for all intents and purposes, IDEs are dead.”

Q For example, how are you provocative in phrasing? Cause I find everyone's like, we look for a team player who's good at communication and shared interests and alignment. And you're like, Fuck off.

A Yeah. Zuck said, um, said something very, uh, very interesting, uh, about like move fast and break things. I think a journalist was asking him about it. And he said, like one heuristic to know whether you're saying something meaningful is that like reasonable people can disagree with it. And the opposite of it is also reasonable. So like move fast and break things could be like move slow and don't break anything. And that could be like a value at IBM. Right. Uh, you know, or steady or slow, you know, you can phrase it like a little more charitably. And so if something where the opposite of it doesn't make sense, or nobody would actually, it's, it's not a value anyone would hold, they're actually not saying anything. Uh, it cancels each other out. And so to be provocative, um, you need to say something that people will disagree with, and the opposite of is also somewhat reasonable. One of our values, for example, is like seek pain. And so the idea behind seek pain is that there's a lot of painful things in, in building a startup. Talking to customers is actually extremely painful. When your product is not working or there's like some fundamental, you know, lack of product market fit or some issue you're dealing with, you know, founders and entrepreneurs typically don't want to face that. And facing that is painful. At Replit, we've, you know, had some things in the past, uh, tha…

AI assessment note: “One of our values, for example, is like seek pain.”

Q I totally get you. Ah, the joys of hindsight. The final one, my friend. Uh, twenty-twenty-eight, so five years time, say. Where's Replit then?

A There's all sorts of ways you can, you can answer that, you know, um, obviously, you know, financial company, all of that, but I, I, I would focus on sort of impact. Like what I want is, uh, I want like a kid in, you know, rural Jordan to be able to have started a startup or successful business. Um, I, I want one, like, people anywhere in the world to, like, be able to come to Replit, teach themselves how to code, go do a few bounties, and generate some income, and then start a startup, and put it up in our, um, app store community, and, like, start generating income from that, maybe start growing it, and I really want that opportunity, uh, to be available for every, uh, everyone in the world. So I want Replit to be a full stack service, From your, like, first line of code, to your first app, to your first company, to your first dollar.

AI assessment note: “I want Replit to be a full stack service”

Q Well, I think you managed to do that with your personal brand. Don't worry. I'm just, you mentioned that kind of like the big risk taker. You said something, uh, before on Twitter that I thought was fascinating. You said great startups are examples of conspiracies to change the world. And I thought, God, that sounds great. What does he mean by that? So what do you mean by that?

A You know, um, I think, uh, if you read a bit of history, you find that most of history is actually Built and changed by like a small number of people. I think we're actually living through a moment of history right now, especially given the AI revolution in, in tech. And, you know, you could argue that open AI is at the center of it. And it's really a small group of people, you know, there are like 300 people now, and they were like much smaller before. Uh, they had a conspiracy and their conspiracy was that if you get a lot of talented people together, that's what the original thing and, and, and you work and really interesting AI research, you're going to find something. And then they found something and, you know, most of the world wasn't paying attention to it, which is if you scale language models, uh, and if you scale, keep scaling them up and up and put more computing data in them, you're going to find that it becomes more and more intelligent to the point that it's actually Reaching some kind of general intelligence, uh, obviously not AGI as, as it's defined, but there's some generality in GPT based models. So that was a conspiracy theory, you know, and that was like a, basically, um, you know, that was like a small group of people with a secret planning for, for a long time. And so they were like conspiring as, uh, as it were. Um, and now they're at a point where they'…

AI assessment note: “that was like a small group of people with a secret planning”

Q software, and you tweeted before the best job description is doing stuff. I find that we live in this generation of planners. They love meetings. They love taking notes. They love planning. Let's brainstorm. Fuck, why didn't you just do something for once? Um, I'm just venting now. Um, so you said the best job description is do stuff. What did you mean by that? Just unpack that for me.

A You know, I, I said that, um, outside of design and engineering, because in design and engineering, you actually can be somewhat specific about what people do. You can hire a machine learning engineer, and, you know, they're going to be training neural networks, or you can hire, um, you know, uh, a, uh, sort of drivetrain engineer at a, at a car company, and they'll be working on that. So, but outside of that, I, I think, and even in engineering, in some cases, People just need to be entrepreneurial and need to get stuff done, right? So at Replit, we actually have, maybe we're not known for that, but we have a very talented sort of business, legal, PM, um, biz ops teams. We're still a small team, but they're actually all very entrepreneurial and they're all just get stuff done. And so like, we get a lot done because they're behind the scenes instead of being the connective tissue of the company. They don't let any ball drop. You sort of like, See them figure out anything and jump into anything and learn it. And, um, and as a CEO, you know, I do that and try to Exemplify that as well.

AI assessment note: “People just need to be entrepreneurial and need to get stuff done”

Q uh, which is always dangerous. Never trust a VC who's an armchair economist. Terrible trades. Um, but like, you know, when you look at simple demand and supply theory, when you actually allow the world to create code, and when you create this access within developer time, you have an access of supply. Uh, will we see the price and the real wages of developers go down as a result?

A You know, Probably like the, I think there are developers today that are commanding insane, uh, amount of salaries while actually working very little. And like, it goes to what we talked about. I'd like the value of hard work and all of that. The idea that you can like go coast at big tech and still be like a millionaire, I think it's absurd. And I think that will go away. I think that's not just a, there's just the effect of an inefficient market, right? And perhaps a zero interest rate market. But, but I, I think that was an unnatural thing that happened. And so in that regard, yes, maybe the average, you know, developer salary would go down, but I think the, the most creative, um, you know, most hardworking developers will actually be able to earn a lot more.

AI assessment note: “yes, maybe the average, you know, developer salary would go down”

Q that causing such controversy, and then you look at the app store and see their three top placed consumer apps, and I'm like, well, they're sucking data from three hundred and twenty million phones, but you're right, we should shoot down a balloon, um, and be worried about it. Um, my question to you is, How do you feel from a security standpoint about Chinese domination in U.S. consumer apps?

A Look, I mean, I, I moved to the U.S. because I, I like, uh, sort of Western values of, of freedom and, and democracy and individualism, and I think the Chinese Communist Party embodies none of that. Uh, like, it's pretty obvious they say it, they don't like individualism, And so, they're clearly at odds with our, with our values. Um, but, but, you know, part of our values is free, free market and free competition. And so shutting them down just because we don't agree with them is also bad and corrosive to future investors and future sort of countries, uh, and other players all over the world that want to invest in the US, right? So like we, we need evidence that they're actually doing something bad, but if we don't have that evidence, I would, I would think it's an, it's government overreach to go and try to ban them.

AI assessment note: “we need evidence that they're actually doing something bad, but if we don't”

Q Final one for you. What do you know now that you wish you'd known when you started Rapplet?

A I mean, the feeling of product market fit is you can deceive yourself into thinking you had product market fit at different points. You'll get a few customers like, oh, this product market fit. Real product market fit. And like, people will say it You can't really internalize it is, is the idea that like the product is getting pulled out of your hand. You can't even provide it fast enough. You know, I think, I think if I understood that earlier on, uh, perhaps I would have searched for it faster or harder or, or things, things like that. So as an entrepreneur wanting to build not a lifestyle business or small business, wanting to build a venture scale business, you have to find that moment. You have to keep pivoting and changing. Perhaps you don't change your vision, but keep changing the different Keep, keep, keep kind of searching until you find that explosive demand.

AI assessment note: “if I understood that earlier on, uh, perhaps I would have searched for it faster”

Q The thing that I always hear is that we're using frontier models to basically set benchmarks. We're seeing where those benchmarks lie, and then we're switching to open source to get as close to them as possible with much more efficiency in terms of cost. Is that the future?

A It depends, right? Like, so cost question, I think is secondary to the performance question, right? Especially in a, in a, in a time when, when it's flush with capital. Uh, I think when you focus on cost is when you reach a certain Sort of, uh, asymptotic, you know, plateau in the S curve, and you don't foresee, um, a massive improvement, specifically in your domain. Like Intercom might say, you know, we, we don't predict that models are going to get that much better on customer support for the foreseeable future. Therefore, you know, we, we can focus on, on building our own Or there's like a data flywheel that we can, we can get, or there's a cost advantage that we get. But if you focus on cost at the expense of performance, you're going to lose. Uh, and so, you know, it's similar to any era in tech, right? Like it's cloud, it's mobile, it's a SAS, whatever it is, there are moments of time where, where the goal is growth and performance and being at the edge. And then when things kind of rationalize or Uber lift, when things rationalize, you, you kind of focused on your gross margins.

AI assessment note: “cost question, I think is secondary to the performance question, right?”

Q When we think about Price sensitivity. How do you think about essentially routing different behaviors where some models are actually okay for certain things and some need the frontier? How do you think about intelligent model selection for different functions?

A I would say that is the core Competency IP of an agent lab, right? If you think of yourself as an agent lab, you have this tacit knowledge first, Of evaluating models. Like, I think about our AI engineers, it's kind of psychologists in many ways. When a model comes out, the first thing they do, they like sit down with it for a day or two, right? They play around with it, they plug it in, they're like, okay, what are the limits? What can it do? And then we plug it, and, and, and that's more like the tacit aspect of it, right? Which is very easy to underestimate. But the reason Replit, when a new model comes out, we're able to build State of the art performance, um, even better than the lab itself. Like we're big partners with Google and Gemini is one of the best models at design. I would say our products are better at design using Gemini than Google's products. It's because we know how to, how to evaluate these models, how to get the best performance out of them. And then we have a bunch of proprietary benchmarks that we use. And finally, we do a lot of AB testing as well. So that's really what you're doing as a company that is building on top of Foundation models.

AI assessment note: “we have a bunch of proprietary benchmarks that we use. And finally, we do”

Q So if you're a student listening to this, should you not study engineering at university? Should you not study CS? How does this inform how you think about advising young people?

A Before 2005, right, let's say, which is when I went to school, uh, people who went to the computer science We're very intrinsically motivated in understanding computer science and understanding exactly how computers work. And there were like hackers and really interested in programming, right? And then after that, computer science became a hyped up field because you can, it's the easiest place to make money, right? And we had the boot camps and we had this whole thing and computer science Uh, departments exploded because of that. Now, if you're not into computer science, if you don't feel like you're drawn to it, like a fly drawn, drawn to a light, then don't go into it because someone told you you're going to make a boatload of money working for Google. That's gone. So that it's pretty dumb to like tell people to go into computer science if they're not really intrinsically interested in it. Now, if you're interested in it, uh, I think there's still Ways to contribute. I think we, you know, there's, uh, you can, you can get into ML and AI and go work at, at like the big labs or a company like ours. You can get-

AI assessment note: “don't go into it because someone told you you're going to make a boatload of money”

Q Enterprises will kind of try things in, in certain respects. You know, I was chatting to Jason Lemkin before, and he said, you're number one ICP's product teams. As product teams kind of code or vibe more, what do they do okay in two to three years? How do they think about those functions?

A There's a, there's a big question about how product teams will look in the future. If I were to make a prediction, I would say that we'll still have engineers inside the organizations. Those engineers are responsible for more infrastructure, AI, ML, uh, embedded systems, you know, more, more, you know, low level engineering. And then you have product organizations and product organizations will have people that are like tilt a little more technical and have people tell it a little more design, have people tell it a little more product. But you're not really calling them anything different. They're like a, they're product builders, and their responsibility is to figure out what to build next. I agree with Jason that a lot of the ICP right now is product, but one thing I'm really excited about is operations teams, and they're kind of underserved. Like, operations teams are sitting at the nexus of a lot of data flow. And typically they'll buy a lot of SaaS software. They're typically not happy with it because there's all these SaaS softwares like styling the data. They try a lot of automation software that doesn't work very well. They have a lot of Excel sheets, a lot of manual work. And so, you know, we see a lot of our customers are building, you know, quote, uh, you know, quote configurators for their sales team, automating their deal desk, um, you know, automatic support, uh, …

AI assessment note: “They're like a, they're product builders, and their responsibility is to figure out what to build next.”

Q Would you have a moral issue with using a Chinese model?

A I don't think there's a moral issue per se. Like, I don't think they're using slave labor or anything like that to, to, to do that. Uh, I would have a security issue, especially since we have enterprise customers that, that depend on us for sensitive, uh, data and things like that. So we haven't taken the step yet. Um, I wouldn't preclude it from taking in the future. I would love to see a U S corporation investing in open source. It looks like NVIDIA is making moves in that, in that regard. But open source is going to be very important for us to actually have a free market around AI, because if we're going to end up in an oligopoly of AI companies, then the, you know, there's actually an economic theory of how they'll naturally collude on price and prices will not go down as fast as possible. They'll also control how we use these AIs. They'll also, uh, not provide everything through the API and keep some of the models for them for themselves. And so, I think that it would be bad if we're in a situation where AGI or AI is only controlled by a few corporations. So open source is going to be very, very important. I would venture to say, like, maybe the US government should start a consortium of companies that are, like, creating the best open source, national open source model, uh, so that the market stays competitive.

AI assessment note: “I don't think there's a moral issue per se.”

Q we start with like, oh, the founding moment of the company, but you have a pretty awesome background. That's not like, oh, I grew up in Silicon Valley, and surprisingly, I founded a startup. Um, such a surprise. Um, but you actually grew up in Jordan, and so I wanted to ask, what were you like growing up as a child in Jordan, and, and how were those early years?

A I was a big troublemaker. I was, um, I looked very different. So I'm like a redhead. Um, my family is actually sort of, um, split right in the middle where some of my uncles are, you know, extremely dark complexions and like a couple of parts of the family, like we're extremely like bright redhead. I'm actually like fairly, um, you know, on the, on the, Sort of not too intense sort of redhead, but we have like really intense sort of, uh, gingers in the family, but I, I look fairly different and there isn't a lot of gingers in Jordan. Um, and at the same time I had a bit of a temper. Uh, I had a bit of, um, you know, a little bit of a chip on my shoulder, sort of rebellious angle to me. And we, we grew up in a, You know, my father was like this, ah, he worked in government and he didn't make a lot of money. He was an engineer, very kind of low level engineer in government. But, um, he really put a lot of money behind our education, and so we went to school in a good part of town, and so coming from a bit of a, let's say, a challenging part of town where you sort of had to fight and you had to, uh, grind and hustle to, to kind of, uh, to just play and, and be just a kid, and then when we go to school, we're like a little bit Odd ones out. And again, like I said, my appearance was a little different. And so I always got in trouble, like nonstop, like since, since I was, uh, six ye…

AI assessment note: “I was a big troublemaker.”

Q When you look ahead, I see the green shoots. I see the creativity. I see the enablement. What, what does concern you?

A Yeah. So wealth inequality will grow insanely a lot. And how so? Because like people look at Instagram or WhatsApp as these outlier companies that were able to achieve massive valuations with like very little, with very little people. And I, I don't think it's going to be an outlier in the future. I think we're going to see a lot of people, a lot of companies, five, 10 people that are going to have massive impact. Probably be valued in the billions of dollars. And I think it's going to create a new crop of millionaires and billionaires, um, in Silicon Valley and elsewhere. Uh, and these people are going to have more and more power in the world. Um, and I think that will create more envy. I think humans will just continue to get richer But I think wealth inequality is concerning, not because it means that, you know, some people are getting sort of the short end of it. Sometimes that happens, but I don't think it happens in technology. I think what happens is just creates more envy because people think they look at, you know, rich people and they think, oh, you must have robbed someone to, to, to, to get there. Um, that's how politicians kind of frame it as well, right? And when you hear Elizabeth Warren and these kinds of people, uh, talk about rich people, uh, Um, they talk about as if you're, as if they're like thieves, right?

AI assessment note: “wealth inequality will grow insanely a lot”

Q I agree with you. There's, like, a scale of, like, well, how far is wrong? Just a little bit wrong? Like, or very wrong? Like, what, what, I, I get you there, but, but I don't, Like, do you think TikTok will be banned?

A Um, I, I think there's going to be increasing pressure on them to move more of their operations to the US. Obviously, they use Oracle data centers now. Maybe there's going to be more pressure to spring up more firewalls there. I think maybe there's more pressure for, um, like more US-based, um, Like companies, maybe investors in the U S have to own more of a company than, than by dance. Um, I, I think things like that will happen just with the increased pressure. And I think TikTok will have to become more transparent because right now they're, they're like, not very transparent. They use it like every growth hack. It's like a very slimy app in that way. Um, but I think they'll, they'll just have to react to the sort of public pressure on them. And I think they might be able to do a good job and sort of stave it, stave it off and give the US government, US people some assurances that they're, they're not, you know, uh, collecting data and sharing with the CCP. Um, And, and I think maybe, maybe that would work out. Banning them outright, I don't see that happening unless there's like a full Republican government and like, you know, will of, you know, political will.

AI assessment note: “Banning them outright, I don't see that happening unless”