Sep 21, 2023 · 29m · no-priors
No Priors Ep. 33 | With Replit's CEO & Co-Founder Amjad Masad
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of No Priors, Replit CEO and Co-Founder Amjad Masad discusses the evolution of cloud-based development, the architecture behind AI coding tools like Ghostwriter, and the impending rise of autonomous software agents. He provides insights into custom model training, full-stack developer moats, and how AI-native marketplaces and programmable money are reshaping the software economy.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 19.8% of the talking time here. How this is scored →
The hosts as informed peer 4.7 Guest teaching 3.1 Guest disagreement 0.7 The hosts pushing back 0.4
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 10:46 Pushing back on data moats and user RLHF
Amjad politely rejects Elad's premise that user volume creates an RLHF moat, arguing that data moats are easily overstated and the true advantage is execution feedback.
Hardest push from the hosts ▶ 17:51 Elad refines the requirements for open source sponsorshipElad pushes past Amjad's characterization of corporate 'guts' to argue that talent scarcity and massive capital commitments are the real gating factors.
Biggest teaching moment ▶ 11:05 Explaining the end-to-end platform telemetry advantageAmjad educates the hosts on why IDE extensions lack vital feedback loops, whereas an integrated runtime, deployment, and crash environment yields far richer training data.
The host holds their own ▶ 18:05 Historical analysis of corporate-backed open source wavesElad demonstrates deep structural industry knowledge by citing historical precedents such as IBM spending $1B a year on Linux and Apple/Google driving WebKit.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| The Genesis and Early History of Replit | 3 | 2 | 0 | 0 | Sarah prompts Amjad on his transition from Facebook to Replit and the inception of Ghostwriter. Amjad shares his technical background compiling languages in-browser and applying early NLP concepts to code. | |
| Next Frontiers and Productivity Leaps in AI Software Development | 4 | 3 | 1 | 0 | Sarah asks about the next productivity frontiers beyond local autocomplete. Amjad explains why static code training is insufficient and why models need runtime evaluation context and environment interaction. | |
| Timeline to Autonomous Agents and the Future of Engineering | 4 | 2 | 0 | 0 | Elad asks for specific timelines on autonomous coding agents. Amjad breaks down near-term infrastructure work within current model limits versus a longer multi-year shift toward engineers acting as AI managers. | |
| The Strategic Value of Replit’s End-to-End Platform | 6 | 5 | 3 | 1 | Elad suggests Replit's 22M users create an RLHF data moat based on recent Google research, but Amjad pushes back that data moats are overrated compared to full-stack execution telemetry. Amjad also criticizes vanity model training and open-source benchmark chasing. | |
| Open-Source AI Ecosystems and Meta's Strategic Role | 7 | 4 | 1 | 2 | Elad asks about the sustainability of Meta open-sourcing Llama, leading Amjad to recount pitching Zuckerberg on an Open Compute model for LLMs. Elad deepens the discussion with historical parallels like IBM funding Linux and corporate backing behind WebKit. | |
| Replit Bounties, Marketplaces, and Autonomous Economic Agents | 4 | 3 | 0 | 0 | Sarah asks how Replit Bounties connects to agentic workflows. Amjad explains how human beginners leverage AI to deliver MVPs cheaply, critiques GitHub stars as fake currency, and outlines autonomous AI bounty hunters. | |
| The Changing Paradigm of Hacker Education and AI Tooling | 5 | 3 | 0 | 0 | Sarah details how college hackers bypass traditional CS hierarchies by orchestrating AI, while Elad asks about crypto versus centralized payment rails. Amjad explains programmable money and staking mechanisms for task SLAs. |
Statements from this episode (20)
Masad: 2012 'Naturalness of Software' paper proved NLP models apply to code
“There was this seminal paper in 2012 called on the naturalness of software. And basically a bunch of researchers try to apply NLP to code. And what they found is that actually code Can be modeled like any language. That's why they call it naturalness is becaus…”
Masad: Machine learning on code did not work effectively until GPT-2/GPT-3
“Every year or two while starting Replit, I would go look at the state of the art on ML on code and nothing ever really worked all that well up until GPT-II. And you can like take GPT-II and fine tune it and like try to make it write code and was like kind of o…”
Masad: Replit users see 30% to 50% productivity boost on coding
“Based on our research of how people are using our products. We think it's 30 to 50% more productivity on coding itself.”
Masad: A non-coder built a $250k ARR startup in months using Replit
“We've profiled a, an entrepreneur That has a quarter million ARR startup in a few months, and they haven't, you know, earlier this year, they didn't know how to code.”
Masad: Professional programmers will see 2x to 10x productivity gains from AI
“But I think if you project forward, those people will also get a lot more tools and going to get more productive, going to hit superhuman productivity. But the professional programmers, this is where actually you start to have that two X, five X, 10 X multiple…”
Masad: Useful AI coding agents are 6 to 18 months away
“My feeling is this is within reach, even in the current capabilities. Like if you hold constant the current capabilities, this is when reach within reaching the order of, you know, six to 18 months.”
Masad: Replit is building container infrastructure to embody AI agents
“Right now we're doing the hard work of building infrastructure on Replit. Like we are building this service that sits inside the container that embodies the AI In the development environment such that it can do things like go to the internet, like install a pa…”
Masad: Programmers will act like managers of AI within 3-5 years
“Like I think software could fundamentally change in the next four, three to five years where it looks totally different to someone sitting, sitting today and looking at it into the future. Maybe it looks something like programmers tend to be more like engineer…”
Masad: Replit's Moat Is Unifying the Entire Code-to-Deployment Feedback Loop
“The real advantage is the end-to-end journey from the first line of code to the deployment, to getting crashes in production, to making edits, to all of that feedback cycle, which typically in most cases is divided between GitHub and VS Code and AWS and all of…”
Masad: Full-Stack Dev Platforms Will Enable AI Training Over Large Action Spaces
“And this is, I think, where we're going to get the kind of Richness of training data that allows you to train over a large action space, and that's really exciting.”
Masad: Commercial AI providers will not release completion models going forward
“Now all the models are chat models. You know, they're not going to be releasing any completion model going forward. And so if you want to build a completion based product, it's actually fairly difficult to do it using commercial APIs.”
Masad: Replit data delivered a 50% improvement over open-source model
“And then we applied some more tokens from replets data that gave us a 50% improvement over the model that we open source.”
Masad: Replit's 3B model outperformed open-source models 20x its size
“And it was state of the art model was better than most open source code models at the time, even things, 20 exit size.”
Masad: Training models has become a founder status symbol over product needs
“I think a lot of founders now see it as a point of pride or status to train a model. And the moment we put out the three billion parameter model, like three or four copycats came out of after that. And I, I'm not a fan of what's happening in, in the open sourc…”
Masad: Pitched Zuckerberg in 2022 on open-sourcing LLMs via Open Compute playbook
“And I got a meeting with Zuck actually, and I pitched Zuck on this idea. And I remember around the time he did the open compute project. So what open compute was like Google and AWS are going to keep all their data center secrets because it's a company advanta…”
Masad: Open-source AI models will eventually match commercial models in performance
“I think on a long enough timescale, obviously there's going to be open source models that are as good as commercial models, but the current landscape makes it hard to kind of imagine how it would play out.”
Masad: Users can buy a startup prototype for $50 to $100
“You can go to Replit and get a startup prototype for startup or an MVP for startup for as little as like, 50 dollars or a hundred dollars in some cases.”
Masad: Fully automated AI bounty hunters will autonomously earn money
“In the future, I could see people creating bounty hunters that are fully automated. So can you create an AI that can go earn money for you while you sleep?”
Masad: The right hacker mindset is just-in-time learning around AI
“I think embedded in their answer is this attitude of like, if it becomes a problem, I will learn it, right? If the AI can't do it, I will learn it. And I think that's the right attitude for hackers is that you learn just in time to be able to do that task.”