Jan 9, 2025 · 23m · no-priors
No Priors Ep. 96 | With Modal CEO and Founder Erik Bernhardsson
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Modal CEO Erik Bernhardsson discusses building specialized serverless cloud infrastructure for AI workloads, the economic transition from static GPU reservations to dynamic multi-tenancy, and why custom model engineering and AI developer tooling will expand the software industry.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 25.3% of the talking time here. How this is scored →
The hosts as informed peer 4.8 Guest teaching 1.8 Guest disagreement 1.3 The hosts pushing back 1.9
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 4:06 Erik correcting Elad on Suno's backbone architecture
When Elad asserts that Suno uses an entirely transformer-based backbone, Erik immediately pushes back that it is likely a hybrid combination.
Hardest push from the hosts ▶ 10:22 Elad challenging multi-tenant adoption in enterprise environmentsElad systematically lists reasons enterprises resist third-party compute platforms, including compliance reviews, existing cloud credits, and data egress latency.
Biggest teaching moment ▶ 11:10 Erik comparing multi-tenant AI skepticism to Snowflake's early daysErik reframes Elad's concerns about enterprise adoption by citing historical precedents where the industry initially doubted both early AWS and Snowflake.
The host holds their own ▶ 20:06 Elad citing theoretical physics and corporate simulation researchElad demonstrates subject-matter depth by discussing string theory directions via Ed Witten alongside specific weather modeling research from Nvidia and Google.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| The Founding Origins and Genesis of Modal | 4 | 2 | 2 | 2 | Elad opens with background on Erik's work at Spotify and Better.com, lightly probing Modal's core architecture. When Elad suggests Suno runs purely on a transformer backbone, Erik politely nuances that it uses a combination of approaches. | |
| Dynamic GPU Allocation Versus Static Capacity Waste | 5 | 1 | 1 | 1 | Elad references Erik's essay on flexible GPU capacity and adds his own perspective on how training runs yield a single weights file. Erik agrees and highlights why inference requires elastic, multi-tenant infrastructure. | |
| Supporting the End-to-End Machine Learning Lifecycle | 4 | 2 | 1 | 2 | Elad inquires about the full end-to-end ML stack and asks how Modal differentiates from competitive platforms. Erik explains Modal's focus on high-code developers, custom scheduling, and custom container runtimes. | |
| Enterprise Cloud Adoption, Data Locality, and Security Paradigms | 6 | 3 | 2 | 4 | Elad challenges the multi-tenant third-party model by detailing enterprise concerns around data locality, egress latency, and hyperscaler lock-in. Erik counters by drawing a historical parallel to initial enterprise skepticism of AWS and Snowflake. | |
| Open-Source Model Trends and the Frontier of Audio AI | 5 | 2 | 1 | 2 | Elad brings up vector databases versus traditional relational systems like Postgres with pgvector. Erik speculates that future AI data storage might natively embed representations directly rather than exposing conventional database interfaces. | |
| Model Training as a Defensible Competitive Moat | 4 | 1 | 1 | 1 | Elad asks about heuristics for training proprietary models versus using off-the-shelf APIs and brings up Erik's competitive programming background. Erik explains why proprietary model quality serves as a defensible moat and argues AI will unlock latent software engineering demand. | |
| AI in Physics Simulations and Computational Biology | 7 | 2 | 2 | 3 | Elad brings up high-level physics theory, citing Ed Witten and weather simulation papers from Nvidia and Google. Erik engages with the deep learning applications in meteorology turbulence and computational biology. | |
| Transformative Impact in Medical Imaging and AI-Generated Music | 3 | 1 | 0 | 0 | Elad asks about broad human impact applications, and Erik shares observations on automated electron microscopy pipelines in medical imaging as well as music generation breakthroughs at Suno. |
Statements from this episode (15)
Bernhardsson: Modal Built Custom File System, Scheduler, and Container Runtime
“We can't really use Docker and Kubernetes. So we're going to have to throw that out and probably going to have to build our own file system, which we did pretty early in a build our own scheduler and build our own container runtime. So that was like basically …”
Bernhardsson: Modal Can Provision 100 GPUs Within Seconds
“We run a very big compute pool, like thousands of GPUs and CPUs, and we make it very easy to get, you know, if you need a hundred GPUs, we can typically get you that within seconds.”
Bernhardsson: Stable Diffusion Release Drove Initial Traction for Modal
“The main thing that really started driving all the traction was when stable diffusion came out. And a bunch of people came to us and like, Hey, actually, this looks kind of cool. Like you have GPU access. It's very easy to, you know, you have to think about, y…”
Bernhardsson: AI Music Startup Suno Runs All Inference on Modal
“So, so one example of a customer I think is super cool building really amazing stuff is Suno. Which does AI generated music. So they run all their inference on modal very large scale.”
Bernhardsson: Long-Term GPU Contracts Are the Wrong Model for Startups
“Means that cloud, you know, a lot of the cloud capacity is like, you know, the only way to get it is to sign long term commitments, which I think for a lot of startups is really not the right model for how things should be.”
Bernhardsson: Modal Is Expanding into Bursty Experimental AI Training
“Traditionally, most of modal has always been inference. Like that's been our main use case, but we're really interested also in training. So in particular, like probably focused more on these like shorter, like very bursty sort of experimental training runs, n…”
Bernhardsson: Dynamic multi-tenant pooling is the future of AI compute
“I think there's so many benefits of this multi-tenant model in terms of capacity management that to me, it is very clearly like a big part of the future of AI is like running a big pool of compute and slicing it very dynamically.”
Bernhardsson: Audio AI is underexplored with massive open-source opportunity
“I think audio is like very underexplored. I think there's a lot of opportunity for open source models in that space. But I don't think we've seen anything really cool yet.”
Bernhardsson: AI-native data storage will take 5–10 years to mature
“The more native, like AI native storage solution would be, you put text in, you put, you know, video in, you put image in, and then you can search by that. Like to me, that would be like a more sort of native, AI native sort of storage solution. So that's like…”
Bernhardsson: Companies reliant on model quality must train custom models for defensibility
“I think eventually like for any company where model quality really matters, unless you kind of train your own model in the end, like, I feel like it's going to be hard to sort of defend the fact that like, You know, you're, you have a better solution. Cause li…”
Bernhardsson: Developer productivity gains historically increase the number of engineers
“Every time that's happened, you know, It turns out like there's so much latent demand for software that actually like the number of software engineers goes up. So like, I feel like you look back at like, you know, last like four years of software development, …”
Bernhardsson: AI will expand rather than destroy demand for software engineers
“So I, I'm very bullish on software engineers. I think it would take a lot to sort of destroy that demand. I think people look at a lot of like AI as like a kind of fix something, but in my opinion, it's like, No, it's just gonna unlock more latent demand for m…”
Bernhardsson: Deep learning should transform meteorology and turbulence modeling
“Meteorology is like something I actually think, like, deep learning should, like, change, right? Like, it sort of makes a lot of sense. Like, you know, deep learning should be very good at, like, you know, predicting, you know, turbulence and things like that.…”
Bernhardsson: Modal sees growing platform usage in computational biology
“And that's actually a field where we start to see a lot more usage as model as well. Is there's a lot of, I feel like there's like a kind of a resurgence of computational biology.”
Bernhardsson: Music is historically among the first sectors transformed by new tech
“Music in itself tends to be like sort of always like one of the first areas where you see Real impact of new technologies, whether, you know, Spotify or like iTunes or piracy or like all these things or gramophones going back.”