Switching Costs for Open-Source LLM Inference Providers Are Zero
Erik Bernhardsson · Truly Serverless Infra for AI Engineers - with Erik Bernhardsson of Modal · Feb 19, 2024 · at 49:55
Modal founder and CEO Erik Bernhardsson discusses the economics of cloud infrastructure and the price wars among open-source LLM inference providers.
0:00 / 0:17exact quote · 17.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The LLM space is, like, the opposite. Like, the switching cost of LLMs is zero, right? Like, if all you're doing is, like, straight up, like, at least, like, open source, right? Like, if all you're doing is, like, you know, using some, you know, inference endpoint that serves an open source model, and, you know, some other provider comes along and, like, offers a lower price, you're just gonna switch, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Erik Bernhardsson
AI Productivity Gains Will Ultimately Increase Demand for Software Engineers
“Like, I think we can, you know, 10 x the amount of developers, and still, you know, have a lot of people making a lot of money, you know, building amazing software, and also being, while at the same time being more productive. Like, I never understood this, li…”
VCs Subsidizing AI Inference Will Not See Their Expected Returns
“In the end, like, I don't think VCs will have the return they expected. Like, you know, in these things, but guess who's going to benefit? Like, you know, it's the consumers, right? Like someone's like reaping that the value of this. And that's, I think an ama…”
Startups Must Build Deep Infrastructure Rather Than Thin Kubernetes Wrappers
“And so one of my frustrations has been like so many startups are like, in my opinion, like Kubernetes wrappers and like, you know, like, and not very like thick wrappers, like fairly thin wrappers. And I think, you know, every startup is a wrapper to some exte…”
Kubernetes Works for Backend Teams but Fails Data Teams
“Like in particular, like Kubernetes, I feel like it's like kind of worked okay for backend teams, but not so well for data teams.”
AWS Has Always Struggled With Developer Experience
“AWS has always like struggled with developer experience”
Developer Productivity Will Continue Growing 10x Per Decade
“You look at the developers are, I've been getting like probably 10 X more productive every decade for the last four, four decades or something that was kind of crazy. Like on an exponential scale, we're talking about 10 X or is there a 10,000 X like, you know,…”