Feb 28, 2025 · 1h 1m · latent-space
Browserbase: Browser Infrastructure For Your AI Agents
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of the Latent Space podcast, Browserbase founder Paul Klein discusses building dedicated headless cloud browser infrastructure to power autonomous AI web agents. He explores the engineering hurdles of running Chromium at scale, introduces the Stagehand framework, and outlines the emerging paradigms of agent authentication and browser virtualization.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.4 Guest teaching 3.8 Guest disagreement 1.5 The hosts pushing back 2.0
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 48:10 Pushing back on full OS virtualization for agent computer use
Paul offers a sharp, opinionated take dismissing full OS virtualization startups, arguing headless browsers deliver 90% of UI automation capability at a fraction of the cost.
Hardest push from the hosts ▶ 52:07 Demanding falsifiable predictionsSwix refuses to accept Paul's broad vision of the future of software, pointedly pressing him for concrete, falsifiable bets with skin in the game.
Biggest teaching moment ▶ 12:43 Masterclass on stateful browser infrastructure pitfallsPaul educates the hosts on the layered complexities of running Chromium in production, explaining font rendering, container sizing, and stateful socket management.
The host holds their own ▶ 17:45 Technical critique of serverless Fargate limitationsSwix leverages his deep infrastructure background to probe Fargate performance limitations, prompting Paul to confirm why lower-level Firecracker microVMs were necessary.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Welcome, 1-Year Milestone, and Browserbase Origins | 4 | 2 | 1 | 1 | Swix and Alessio warmly introduce Paul Klein, highlighting his previous ventures and Browserbase's status. Paul explains why running headless browsers in the cloud requires dedicated containerized infrastructure rather than simple serverless Lambdas. | |
| AI-Driven Web Scraping vs Traditional Integration Testing | 6 | 4 | 2 | 2 | Alessio draws on his early engineering background in integration testing to ask what makes AI browsing distinct from legacy tooling like Selenium. Paul breaks down dynamic web scraping, JavaScript hydration on modern sites like Airbnb, and LLM-driven non-deterministic automation. | |
| Modern Developer Experience and the LLM OS Primitive | 6 | 3 | 2 | 3 | Swix recalls his initial hesitation about market incumbents and debates whether point solutions or all-in-one platforms dominate the LLM OS primitive. Paul contrasts Browserbase against legacy PE-backed competitors and details the need for modern, Vercel-like developer UX. | |
| Presentation, Aesthetic Polish, and Developer Trust | 5 | 5 | 1 | 1 | Swix asks about the hardest technical challenges among Browserbase's features. Paul walks through the architectural nightmare of running stateful Chromium on AWS ECS/EKS clusters, font dependencies, and video recording observability. | |
| Firecracker MicroVMs and Scaling Serverless Browsers | 7 | 4 | 1 | 3 | Swix probes the promise of millisecond browser spin-ups and questions why AWS Fargate was insufficient. Paul explains their internal architectural shift to low-level Firecracker microVMs to avoid vendor outages on the critical path. | |
| Intelligent Proxy Super-Networks and Ethical Sourcing | 5 | 4 | 1 | 2 | Alessio asks whether proxying or browser orchestration is harder at scale. Paul explains the shady nature of raw web proxy markets, SOC-2 compliance, and how Browserbase maintains a redundant super-network across multiple providers. | |
| CAPTCHA Solving Challenges and Agent Authentication | 6 | 5 | 2 | 2 | Paul addresses the mechanics of CAPTCHA solvers and predicts that internet bot verification will shift toward agent authentication protocols rather than perpetual CAPTCHA arms races. Alessio compares DNS-level Cloudflare defenses with browser-level approaches. | |
| Agent Experience and Identity Protocol Delegation | 6 | 3 | 1 | 2 | Swix analyzes the Agent Experience (AX) concept and delegated authentication. Paul draws on his Twilio IAM background to explain OAuth-like scopes for AI agents to circumvent password sharing. | |
| Live View iframes and Human-in-the-Loop Web Automation | 5 | 4 | 1 | 1 | Swix asks about the technical implementation of Browserbase's live view iframes. Paul details how Chrome DevTools Protocol screencast streams allow human-in-the-loop intervention for 2FA and fallback inputs. | |
| Cloud Infrastructure vs Consumer Desktop-First Agents | 6 | 3 | 2 | 3 | Swix and Paul debate desktop-first consumer browser agents (Dia, Project Mariner) versus cloud infrastructure, followed by a discussion on browser state branching and Linux container forks. | |
| Stagehand Architecture: Observe, Extract, and Act | 4 | 5 | 1 | 1 | Alessio introduces Stagehand's three core primitives (Observe, Extract, Act). Paul explains how Stagehand functions as an open-source natural language superset of Playwright to assist custom agent loops. | |
| Stagehand Positioning in the Web Scraping and Testing Stack | 6 | 4 | 2 | 3 | Swix frames Stagehand as an unbrittle integration testing harness and points out potential overlap with extraction startups like Firecrawl. Paul clarifies that Stagehand targets robust automation rather than high-throughput scraping. | |
| OpenAI Operator vs DOM-Anchored Computer Use | 5 | 4 | 2 | 1 | Paul compares OpenAI Operator's vision-based coordinate clicking with DOM-anchored execution, arguing that direct DOM anchoring offers superior reliability across varied viewports. | |
| Open Operator and Open-Source AI Agent Reference Architectures | 6 | 4 | 2 | 2 | Alessio asks about Browserbase's open-source strategy behind Open Operator. Paul clarifies it is a reference architecture rather than a consumer product, then outlines their cost waterfall for scraping requests. | |
| Comparing Primitives: Lightweight Browsers vs Full Operating Systems | 7 | 4 | 2 | 2 | Swix maps out the competitive landscape of developer tooling primitives (VMs, search APIs, browsers). Paul delivers a spicy take that running full OSs for UI automation is an expensive overkill compared to lightweight headless browsers. | |
| The Future of Software: Software Operating Software | 5 | 3 | 2 | 4 | Paul shares his thesis on 'software operating software' across asynchronous background workflows. Swix humorously pushes back, demanding concrete, falsifiable predictions rather than generic optimism. | |
| Solo Founder Advantages and Rapid Decision-Making | 4 | 3 | 1 | 1 | Swix highlights Paul's status as a solo founder in an ecosystem that often mandates co-founders. Paul explains how operating as a 'benevolent dictatorship' eliminates internal alignment bottlenecks. | |
| Building High-Agency Culture and Hiring Ex-Founders | 5 | 4 | 1 | 2 | Paul discusses maintaining a binary in-person office culture without turning into a grueling 9-9-6 environment, emphasizing the hiring of ex-YC CTOs and former founders for high-agency execution. |
Statements from this episode (37)
Klein: Browserbase grew to 20 people and Series A in year one
“Like, BrowserBase is actually pretty close to our first birthday, so we are one years old, and going from, you know, Starting a company as a solo founder to, you know, having a team of 20 people, you know, a series A, but also being able to support hundreds of…”
Klein: Cloud browsers require stateful containers, not serverless Lambdas
“So if you want to run a web browser or thousands of web browsers, you can't just spin up a bunch of Lambdas. You actually need to use a secure containerized environment. You have to scale it up and down. It's a stateful system.”
Klein: Browserbase spent zero dollars on AI in its AI Grant batch
“I'm in AI grant, you know, browsers was an AI grant, but we were the only company in that AI grant batch that used zero dollars on AI spend.”
Klein: Vision models, not HTML text parsing, drive computer use AI
“Maybe my hot take is like, I was always kind of like, I didn't think vision would be as big of a driver for UI automation. I felt like, you know, HTML is structured text and large language models are good with structured texts, but It's clear that these comput…”
Klein: Browserbase's PE-backed rival hasn't updated its dashboard in five years
“Our biggest competitor actually is partly owned by private equity, and they just jacked up their prices quite a bit, and the dashboard hasn't changed in five years”
Klein: Headless browsers are a core primitive for future LLM operating systems
“Browsers are a core primitive of infrastructure for the LLMOS that's going to exist in the future”
Klein: Developer tools founders should partner with consumer-focused design agencies
“I highly recommend for developer tools founders to work with consumer agencies because they end up building beautiful things and the Parisians know how to build beautiful interfaces.”
Klein: Developers evaluate tools by design polish and trust in the founder
“As a developer, you think you evaluate a company based on like the API reliability and the P-ninety-five, but a lot of developers say, is the website good? Is the message clear? Do I like trust this founder I'm building my whole feature on?”
Klein: Stateful distributed browser systems are painful to run and maintain
“Those are some of the worst systems to run because when it's a stateful distributed system, it means that you are bound by the connections to that thing. You have to keep the browser open while someone is working with it, right? That's just a painful architect…”
Klein: Browserbase briefly became the largest regional Fargate customer before moving lower-level
“We used to be a giant Fargate customer. Actually, the first version of BrowserBase was ECS and Fargate. And I unfortunately it's a great product. I think we were actually the largest Fargate customer in our region for a little while.”
Klein: Infrastructure startups require deeper control than managed services like Fargate provide
“And unfortunately it just, it's a great product, but I think if you're an infrastructure company, you actually have to have a deeper level of control over these primitives. I think it's the same thing is true with databases.”
Klein: Purchasing commercial web proxies online feels like buying illicit drugs
“It's like you're buying drugs when you're trying to get a proxy online.”
Klein: Authentication, not CAPTCHAs, will be the biggest barrier for AI agents
“I actually think that off is the biggest thing that will prevent agents from accessing stuff, not captures.”
Klein: Authentication providers will offer dedicated login features for AI agents
“I think there'll be agent off in the future. I don't know if it's going to happen from an individual company, but actually authentication providers that have a You know, hidden login as agent feature, which will then you put in your email. You'll get a push no…”
Klein: Founders from Stytch, Clerk, and Okta Ventures invested in Browserbase
“And, you know, three from Stitch is an investor, Colin from Clark. Okta Ventures, all investors in BrowserBase because like, I hope they solve this because it'll make BrowserBase's mission more possible.”
Klein: AI agents will use existing web interfaces instead of rebuilding APIs
“The thing that's hard is like reinventing the internet for agents. We don't want to rebuild the internet. That's an impossible task. And I think people often say like, well, we'll have this second layer of APIs built for agents. I'm like, we will for the top u…”
Klein: OpenAI Operator uses VNC rather than Chrome DevTools Protocol
“Operator actually uses a different approach. They use VNC.”
Klein: Web agents still require human-in-the-loop workflows for CAPTCHAs and 2FA
“Web agents need human the loop type workflows still. You still need a person to interact with a browser and building a UI to proxy that is kind of hard. You may as well just show them the whole browser and say, Hey, can you finish this up for me? And then let …”
Klein: Consumer AI agents and tools like Anthropic MCP will run locally
“There's a case where a lot of people are going desktop first for, you know, consumer use. And I think Claude is doing a lot of this where I expect to see, you know, MCP is really oriented around the Claude desktop app for a reason, right? Like I think a lot of…”
Klein: AI apps requiring browser interactions need dedicated backend cloud infrastructure
“If you're building applications that rely on a browser and it's more part of a greater like AI app experience, you probably need something that's more like infrastructure, not a consumer app.”
Klein: Browser state branching and forking is on Browserbase's product roadmap
“And hopefully find the right answer and then say, okay, this was actually the right one and memorize that and go there in the future on the roadmap for sure.”
Klein: Browserbase's Stagehand operates as a superset of Playwright
“Stagehand exposes three APIs, and it's a superset of Playwright.”
Klein: Agent loops belong at the application layer, making Stagehand a framework
“Stagehand isn't a web agent. It's a framework for building web agents. And we think that agent loops are actually pretty close to the application layer because every application probably has different goals or different ways it wants to take steps.”
Klein: There are no genuinely good generic AI agent frameworks
“I don't think I've seen a generic, and maybe you guys are the experts here, I haven't seen like a really good AI agent Framework here. Everyone kind of has their own special sauce, right?”
Klein: Stagehand is built for AI agent reliability, not dedicated web scraping
“If you're building a scraping company, StageChan's not your thing. You probably want something that's gonna, like, get HTML content, you know, convert that to Markdown, query it. That's not what StageChan does. StageChan's more, more about reliability.”
Klein: DOM-anchored agent actions are more reliable than vision-based coordinate clicking
“I think that using mouse click or mouse coordinates, I've seen that proved to be less reliable than I would like. And I just wonder if that's the right form factor. What we've done with our framework is Anchor it to the DOM itself. Anchor it to the actual item…”
Klein: OpenAI's brand constraints will prevent them from offering CAPTCHA solving
“I think it's going to be really hard for a company like OpenAI to do things like support CAPTCHA solving or like have proxies. Like, I think it's hard for them structurally. Imagine this New York Times headline, OpenAI CAPTCHA solving. Like, that would be a pr…”
Klein: Web scraping workflows should use a tiered waterfall architecture
“If you're building web scraping workflows, what you should do is have a waterfall. You should have the first request is a curl to the website. See if you can get it without even using a browser. And then the second request may be like a scraping specific API. …”
Klein: Complex developer primitives like web browsers require bespoke, dedicated infrastructure
“For tools and primitives with sufficient levels of complexity, you need to have a solution that's really bespoke to that primitive, you know, and I am sufficiently convinced that the browser is complex enough to deserve a primitive.”
Klein: Browserbase delivers 90% of computer-use functionality at 10% of OS cost
“BrowserBase can run browsers for way cheaper than you can if you're running a full-fledged OS with a GUI, you know, operating system. And I think that's just an advantage of the browser. It is like, browsers are like little OSs, and you can run them very effic…”
Klein: The future of software is software operating other software autonomously
“The future of software is software using software.”
Klein: Browserbase will be a billion-dollar company within five years
“I can predict that Browserbase will be a billion dollar company one day. So let's check back in five years.”
Klein: Having co-founders slows startup execution by adding an extra alignment layer
“If you want to move extremely fast as a company, one of the hard parts about having co-founders is that there's like, you have to do the co-founder alignment and then the company alignment. And then there's people on the team that probably tell things to one c…”
Klein: Browserbase operates fully in-person Monday to Friday from 10 to 6
“Culturally, like we're fully in-person. Maybe that's one crazy take that we do, but we're also like not too in-person. Like our first meeting's at 10 a.m. People leave around five or six. We work Monday to Friday in-person, and those like, that's the expectati…”
Klein: Former YC CTOs and ex-founders make the best early startup hires
“Culturally, we're a lot of like former YCCTOs and like ex-founders or future founders. And I've just found that those people tend to be just really great early hires for a company. They get it.”
Klein: Polarizing binary decisions make maintaining company culture easier to assimilate
“When you make people make a polarizing decision of like, I will come to an office five days a week, people then end up making more decisions that are aligned with the culture. So it's almost like if you can make your culture binary or you're in or out, It beco…”
Klein: Automating niche local public records extraction offers high-alpha investment signals
“There's so much money to be made in taking like proprietary research and then turning that into like an automation, which is obviously like a very like browser based inspired one, like listening to all of the city halls or town hall meetings in like little tow…”