Modal

“And then if you one day need a thousand GPUs, we can get you, we can typically get you a thousand GPUs like pretty quickly, like talking minutes.”

“In order to get to sort of latencies below, you know, a 152 hundred milliseconds, we probably need to split that up and run like a decentralized control plane.”

“Suno which is AI generated music. And, you know, they have a big cluster. They train their own models outside of modal and then they use modal for the inference side.”

“I actually like them a lot. We use them. Big fan.”

“So it's sort of always felt to me like, you know, in, in hindsight, it looks like the right idea, but applied to the wrong problems. And I always like wonder, you know, now, like I'm looking at like what we built at modal. Like I I'm kind of convinced that lik…”

“Modal makes it easy to build, scale and deploy applications in the data, AI, machine learning realm. So, so we basically, you can think of as like, we take, you write a little bit of Python code, and we take that code, we stick it in a container, we execute th…”

“Like, so, I don't know, I'm a big believer that, like, you know, in the long run, like, people will train a lot of different custom models.”

“Model inference is probably the majority of our use case.”

“We actually have a system that continuously looks at cloud pricing and solves an optimization problem to sort of, you know, figure out How do we allocate capacity across like, you know, a hundred different regions in order to deliver the capacity we need to ou…”

“So we built our own, we threw out Kubernetes out the window. We threw out Docker out the window. We built our own File system in order to optimize for how container images are distributed. We built our own scheduler in order to maintain this pool of workers an…”

“It puts a lot of different, you know constraints on, on the infrastructure that we don't support today. It's something we're interested in, like, you know, somewhat like, you know, looking at down the road but it's not like a super high priority for the busine…”