The Exchanges

Q Replit or, or REPL. What, what is REPL in, in REPLit?

A You know, uh, for example, a DOS or like a UNIX command line is a form of a REPL. So, uh, it comes from, um, Lisp actually in the 19 fifties at MIT, um, where, uh, you can create, uh, the simplest programming environment with like one command, which is read, eval, print, loop. So read reads the command from the command line. Eval, evaluates it, like a lisp string, uh, print prints out the results, and then goes back to the start. So this is like the simplest IDE in the world, right? Um, and so when I was building, so it, you know, the, the way sort of Repl.it was born, I, I was going to school in, uh, in Jordan, uh, studying computer science and, um, I, I didn't have a laptop. So every time I would want to do some homework, I would have to reinstall the development environment. And that was like the, you know, it was so annoying, right? You have to download gigabytes of software and packages and, and there's always something that goes wrong. Um, and so I was like, you know, I'm, I'm using everything in the browser. At the time, Google was putting docs and Gmail in the browser.

AI assessment note: “one command, which is read, eval, print, loop.”

Q And had you graduated? At some point, you were at Yahoo in Jordan, right?

A Yeah, so I, at that time, I had graduated, yeah. My, my time at the university was pretty tenuous because, um, I just like, I didn't like Sitting in class all that much. Initially, um, I also, you know, I'm sure we'll get into more into that, but I, I was, um, I was reading a lot of sci-fi. I was like really into AI and really into, you know, how do you build a, build a brain, and I was like reading a lot about cognitive science and neuroscience. Um, there wasn't a way to study that in Jordan, but that was like a personal interest of mine, and I felt like we're on the cusp of like, You know, solving intelligence, and I felt like computers will be able to program themselves. So I, so I was like, okay, if computers can program themselves, so what, what can I do with computers? And I was like, okay, well, someone still needs to build these computers and run them. So I went into electrical engineering, computer engineering, as opposed to programming. Uh, and then like two or three years in, I'm like, you know, um, I don't, I don't think machines are gonna be coding themselves anytime soon.

AI assessment note: “Yeah, so I, at that time, I had graduated, yeah.”

Q Assuming that any run of the mill A programmer with Replit, with coding agents, can create amazing applications, then what's the next bottleneck? Is it creativity? Is that, what is it?

A Well, I think, I think if you think about it as a sort of a, as like a factory line or a pipeline, um, you know, uh, the bottleneck has always been sort of like the, uh, the making of the thing, right? So you have an idea, um, and, uh, you need to make, make the thing, uh, and then you need to sort of sell it and distribute it. And we have all these, Ways where sales and distribution has gotten easier over time, like PLG is an example of one. Um, ads, you know, things like that. So there's a lot of innovations and ways in which companies can grow fast today. Um, so the bottleneck is like, oh, how do you, you know, how do you create the thing? And even if you have the capital, it's actually still quite hard to make things, right? And you know that as an investor, sometimes you invest in a company, it's still not working. You can give it as much money as you can. I mean, What is it? Like all these companies, um, Magic Leaps and Quibi or whatever, and like, uh, there's a ton of examples of companies that sucked up a lot of capital and went nowhere. Um, but, you know, uh, if, if, in fact, uh, software creation gets dramatically easier and, and better, then I think, uh, the bottleneck will shift somewhere else. Uh, and my, uh, Prediction, ah, is that the bottleneck will become on the idea side of things, which is counterintuitive because the, um, Silicon Valley, um, dogma is like, i…

AI assessment note: “my Prediction, ah, is that the bottleneck will become on the idea side of things”

Q You are a self-described civilizationist? Yes. I'm not even sure I can pronounce the, the word. Uh, what, what is, what is that, and why is it, um, Important to you.

A Yeah, I've always kind of struggled to kind of describe my, my worldview or outlook, and the, I think the best way I could, I could think about it is like, um, I'm like, in awe of civilization, and I feel like if you restart the world, um, you know, a hundred times, perhaps, uh, only the minority of time will get to the level of civilization that we're in. Um, you know, it's, it's quite hard, uh, you know, to go from a single, uh, cell organisms to, to, to, to multiple cell organism, and it's quite hard to get to more complex, and there's all these ways in which, uh, evolution could have stopped, but I do think no one talks about how, you know, you could have, you know, human civilization could have stopped at, um, sort of the stone age or the agricultural age. There's no reason for it to actually continue developing further.

AI assessment note: “I'm like, in awe of civilization”

Q And how do you think in the general context about, um, evaluation? Uh, so, you know, for hallucination and all the things, but I can, in general, like evaluation is like the, the big problem of those, um, Of, of one of the big problem of AI engineering. So where does that fit there?

A So software agents, um, There's a, there's this benchmark called, uh, SWE bench. So software engineering, uh, bench. Uh, and that's what everyone's competing on. Uh, we don't really compete on that. And the reason is because we are building for a different audience. The people who are building SWE bench, um, are building agents that go from, uh, issue or GR ticket to a PR request. What we go from is a high level product description to an app. And that's like a fundamentally different problem. And so the, uh, the tough thing for us is that there isn't academic benchmarks, which gives you a headstart, right? So we've had to build our own benchmarks. And, you know, as you get users, you get more data and you're, You're able to get some of that data and create benchmarks from it, and, and that's like a quite a, a tough problem. So we have our own internal evals and benchmarks.

AI assessment note: “So we have our own internal evals and benchmarks.”

Q I heard you somewhere talk about, um, ACI versus HCI. Can you go into that? That was, uh, basically what you learned about interacting with machines versus, uh, humans.

A Yeah, so, uh, when we were building, uh, the Rap-Load environment for humans, um, you, you were like, okay, this is the editor, this is where you go and type code, this is the UI for the editor, there's the line numbers, and this is what you see, there's the console, this is where you run the code, and here's the UI for the, for the terminal and everything, here's the files, and here's the UI, here's how the files work, and And then you want to deploy, or you want to add a database. Here's how you do it. Here's how you provision the database. Here's the UI for it. Here's the interaction, right? Um, when you start building, uh, something like the agent, your expectation is like, okay, I'll just present all these things as APIs, and I'll just work. What you find pretty quickly is that If you just do that kind of verbatim, you have, you don't have the best performance. And, uh, there was a paper, uh, I think from like OpenDevin or some of the people that are working on agents in open source were introduced the terms ACI. So the term HCI is human computer interaction. ACI is like agent or AI computer interaction. And the main observation from that paper is that actually you need to create tools that has, like, that almost has a UI, but a UI built for language models. So the, the interesting philosophical thing here is that language models are actually kind of human imitation machin…

AI assessment note: “ACI is like agent or AI computer interaction”

Q Inevitable question, and to the, you know, point about clickbaity headlines is, um, Where, what does all of this mean for professional software developers, right? If we build agents that can empower non-technical people to build fully functioning apps, what is the future for software developers?

A I, I think the best way to think about it is to think about other industries where, where that happened. Um, you know, right now it's Very easy to take an amazing photo with an iPhone. Right. Um, and that a lot of like, you know, I mean, Apple did this whole campaign around, you know, took it with an iPhone. There are a lot of people that are just like making really great content with just, uh, iPhones. Um, professional photographers like didn't die. Maybe the population is not growing as fast, but, but they, they exist and they have their own equipments and, um, it, it tends to be better. I don't know if the sort of the market size of professional photographers have like shrunk, uh, but it's kind of probably stayed constant. Um, and I think that's probably what's going to happen to, uh, software engineering. The growth that we've seen in computer science, um, over the past 10 years, I think will, will slow down, uh, as the, um, need for them to, Uh, it slows down because companies will be more efficient. Suck was on Rogan recently, and he talks in 20, 25, we're going to have a mid-level software engineer equivalent of a coding agent that's going to be able to commit in code. I think it's a little overstated, but, um, I don't think he's, he's wrong. I think we're on this trajectory where you can do a lot of sort of junior to mid-level engineering tasks, uh, Uh, automatically, a…

AI assessment note: “So the need for software engineering will, will, will go down.”

Q like, indeed, uh, you know, if you can build any software, then what does that mean for entrepreneurship? What does that mean for venture investment? Uh, does any company have any, uh, moat, at least from a technical standpoint? Uh, and then, you know. Is that a good thing? Is that a bad thing? It's, it's so huge as a concept that it's hard to wrap your mind around it.

A I feel like venture, uh, capitalists are already intuiting that the, um, sort of golden age of, of SaaS is on its way out. And, uh, you can look at the, uh, sort of the American dynamism trend, uh, like trend to going to the world of atoms away from bits. Uh, you, you have the, you know, after end roll, like defenses, Is a big thing that all these VCs are getting into, uh, government in general, obviously, like, AI, AI chips, um, and, uh, sort of what we used to call hard tech is, is becoming trendy, and, and maybe it becomes the majority of, of venture capital, uh, venture capital investment as opposed to the, to the other way around. Um, uh, I, I think there are still, like, Network effect businesses and software that are, are quite defensible. Like, even if you generate Facebook, you know, you're gonna have to get a billion users, right? So, um, you know, consumer, social, um, those will probably become more, more important as categories for, for VCs. Um, and perhaps there's, there's, there's probably going to be a consolidation of Of, uh, venture capitalists, especially there's like a bit of a squeeze where like the, uh, you know, um, the, a lot of the early stage rounds are like, there's like a lot of angels and solo, solo capitalists that are able to do those. And, and the later stage rounds, they're like so expensive and, and capital intensive that like only a few could …

AI assessment note: “In terms of entrepreneurship, the other thing that you brought up, I actually think”

Q And you mentioned Android, but that, that was your next stop, uh, when you joined Facebook, you joined the Android team?

A Yes, so, um, I was inspired by a lot of the open source work that, uh, Facebook was doing, like, around React.js, and I was very early to that, but I was also inspired by, uh, Zuck's vision for internet.org. Do you remember that Mm-hmm.? Yeah, so the idea, which, you know, Elon ended up doing it, but the idea is, like, Beaming internet from the sky, essentially. Um, and, uh, and again, this, like, fits into this vision that I had of, of, you know, more people connecting to the internet, and, um, and I, I felt that was, like, such an important thing. Um, and, um, and so, when I went to Facebook, I was like, ok, I'm either gonna work, uh, on internet.org, or, um, Or work on, on mobile. So when, when I got there, uh, you know, I, the, uh, the way Facebook worked is like you go through bootcamp, uh, for like three months, and then you do team selection. Cause I didn't know what I was gonna work on, but I picked up Android tasks, um, and, um, did, did really well at them, and, uh, I, I picked really the hardest tasks that you shouldn't, Pick as a, as a sort of a, like a boot camper. Uh, at some point I broke the Android app for like ten million, uh, ten million users, the beta users, and, and, and everyone was surprised that I was able to, to do that as a, as someone bootcamp, because I was taking a lot of risks to, you know, move fast and break things. Um, and, uh, but, but I could…

AI assessment note: “Yes, so, um, I was inspired by a lot of the open source work”

Q And from a, from a tech and sort of Silicon Valley, uh, perspective, what, um, uh, what do you think is important?

A Yeah. Um, Uh, I, I think, I think, you know, without a stable world, there's no tech, and I think, uh, if you want techno-optimism, you have to really care about geopolitics, you have to care about a stable world, because, because, you know, um, it's, it's all connected, you know, that, that's the world that we built. It is a global world. It is globalists, right? So you can't return to this idea of like, oh, you know, I'm a big fan of re-industrializing the U.S., but like, The dependence on China is, is a reality. And, um, and there's no reason for them to be really an adversary. Um, and, and so in terms of like, uh, the Trump's, uh, uh, kind of view on AI, um, I think, uh, you know, just, you know, they already, they already took a hammer to the regulate, to regulations with the executive orders. Um, I think the Biden administration was heavily influenced by the, Sort of effective altruism movement. And they, you know, spent a lot of money on lobbying and everything. And, um, they were really trying to pause or slow down AI development that resulted in the executive order. And I think a lot of other things that the Biden administration was excited about doing. And, uh, you know, now we have Project Stargate, uh, five hundred billion dollars.

AI assessment note: “without a stable world, there's no tech, and I think, uh, if you want techno-optimism”

Q Amazing. And so did you rip and replace everything that you had done with, uh, CodeGen and your own model with Sonnet 3.5?

A We actually, um, stopped caring and perhaps removed a lot of the features of people going to the IDE and coding. It's still possible. But our view and the way we track our metrics now is that when you go into the editor and code, it's almost like the Tesla disengage. Right? Like, so what is the main metric that Tesla tracks? Miles between disengagements, right? And like the V-thirteen, I don't know if you saw the chart, but it goes, you know, it's like, you know, 10 acts V-twelve in terms of like, how many miles to go between, and that's my experience too with Tesla now. Like I go to work without disengaging ones. So when you disengage, they consider it a bug, and they're like, you know, give us feedback. So that's how we're thinking about it. So all the stuff that we built around coding AI, I don't think it matters. What really matters is being able to chat with an agent and get an application on the other end.

AI assessment note: “We actually, um, stopped caring and perhaps removed a lot of the features”

Q And to the government dysfunction and, and all the things, you seem to be, um, very energized by the new administration and that, uh, acceleration. What, what, what is the, um, what is your hope? What, what, um, what does success look like for this new administration?

A So if you look at the Biden administration and if you look at what happened to the world, just look at geopolitics, um, uh, you have the Ukraine-Russia war. Which, you know, I know people are very sensitive about this, but I do believe that, you know, we, we bear some responsibility to, to kind of exaggerating this issue and make it, and like, we made sure that there wasn't, um, any resolution to it, and, and, and, and there was such a, uh, motivation to kind of keep it going. Our relationship with China got, got a lot worse, which is kind of surprising because like Trump, uh, sort of started that, um, And, but, but Biden, like, really took it to the next level, and, uh, and there were moments where they could have gone on the, on the right track, and there's, like, small things where, you know, President Xi visited us, and Biden, same day, kind of insulted him, and, um, uh, and the situation in the Middle East, I think, could have been resolved if Biden was stronger and put some pressure on, on both parties to kind of resolve the situation, and America actually has a lot of power to be able to do that, But, um, but the Biden administration was a combination of both being, um, weak and, um, and so it was just like not having a vision for the world that's, that's coherent. Um, obviously Trump, there's all these downsides to him, but I think, I do think that he's a, uh, powerful …

AI assessment note: “So if you look at the Biden administration and if you look at what happened”