Bernhardsson: Modal discarded Kubernetes and Docker to build a custom stack
Erik Bernhardsson · Can AI Infrastructure Work Like Magic? Erik Bernhardsson, CEO, Modal · Oct 31, 2024 · at 16:45
Erik Bernhardsson, founder and CEO of Modal, explains why Modal abandoned off-the-shelf container tools to build custom infrastructure.
0:00 / 0:20exact quote · 20.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So we built our own, we threw out Kubernetes out the window. We threw out Docker out the window. We built our own File system in order to optimize for how container images are distributed. We built our own scheduler in order to maintain this pool of workers and make it possible to start tasks very quickly. We built our own container image builder because we had our own container image format.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Erik Bernhardsson
Bernhardsson: GPU alt-clouds are real estate businesses adding little stack value
“It's like sort of like almost like real estate, right? Like they own sort of, you know, real estate and they rent it out, arguably sort of, you know, we work, you could say, if you want to be, you know, a little cynical, but like not a ton of value, like highe…”
Bernhardsson: LLM inference compute is shifting toward specialized chips like crypto mining
“I do think it's kind of fascinating how it's like kind of going in the same direction as like crypto mining towards more like specialized hardware.”
Bernhardsson: AI inference spend will dominate model training spend long-term
“It's obvious that like in the long run, like inference spend will dominate just for like the obvious reason that like, that's where you make the money, right? Like, you know, when you're training a big model, that's like, you know, you, you're spending a lot o…”
Bernhardsson: Serverless architecture fits AI workloads better than web backends
“So it's sort of always felt to me like, you know, in, in hindsight, it looks like the right idea, but applied to the wrong problems. And I always like wonder, you know, now, like I'm looking at like what we built at modal. Like I I'm kind of convinced that lik…”
Bernhardsson: Demands for self-hosting mean the product isn't compelling enough
“One trap that I think a lot of startups fall into is that, you know, you go out and talk to a lot of customers and they sort of insist on, on self-hosting. In my opinion, like, you know, sometimes that's actually like a true, like hard concerns, but in many ca…”
Bernhardsson: Lower GPU prices increase value of developer AI software
“If GPU prices were to crash, like the relative value of that software actually goes up. So like, I'm not necessarily sure that like a crash in GPU would be bad for us. I think it actually could be great for us because, you know, if GPU prices go down, there's …”