The Exchanges

Q As any, um, you know, great entrepreneur, you've been very focused, and you seem to have said no to, like, self-hosting kind of scenarios. Where, where do you, first of all, is it correct? And second, um, uh, just walk us through the thinking.

A Yeah. And, and it's just something we're like still debating to a large extent, right? And by the way, I think of it like self-hosting and not just, you know, kind of a binary thing, it's kind of a spectrum, right? There's like, there's like on-prem on one side, there's like sort of, you know, fully multi-tenant on the other side, and there's a bunch of stuff in the middle. There's, you know, VPC peering, private link, cloud prem, you know, BYOC, ring your own cloud. So, so, so, so it's actually kind of more of a spectrum. Uh, but, but we've been sort of cloud maximalists and we've been all in on like, let's just build like a multi-tenant service where we just run everyone's code in our cloud environment. And, and to be clear, like a lot of customers is not, you know, a lot of potential customers is not, you know, comfortable with that model. I tend to think about that as like the cloud itself, you know, I'm old enough to like, I, I, I, I, you know, started my career pre-cloud and I remember the first time I heard about AWS, you know, in 2007, 2008 or something when they launched, like my first reaction was like, How could anyone ever run their code in someone else's computer? That's nuts. Right. And then like just a few years later, I was like doing it myself and I was like, this is awesome. Uh, and, and so, and then like, I think there's like sort of a similar thinking, you k…

AI assessment note: “we've been sort of cloud maximalists and we've been all in on”

Q What are you building next in terms of a roadmap you can share?

A I mean, like, I, I hope to build this for the next 20 years. So like, I think there's so much stuff we want to build. Like right now, like we're, we're focusing a lot on just like taking the existing compute platform and, and improving it in various ways. Like one thing, for instance, like I'm, I'm very focused on like, How can we get into more like real time, like low latency use cases, which is like kind of an annoying, you know, annoyingly complex technical problem. Like we, we've sort of, you know, relied on like a centralized control plane running in one single region, US East one. Uh, but we sort of recognized in order to get to sort of latencies below, you know, a 152 hundred milliseconds, we probably need to split that up and run like a decentralized control plane. So that's going to take a long time.

AI assessment note: “run like a decentralized control plane”

Q You mentioned Luigi a minute ago. Uh, what was your journey into, uh, model? What did you do before? Uh, and what was the path to starting the company?

A Yeah, so I was, I've done a bunch of different things, but in particular, I was at Spotify for seven years, and I built a thing called Luigi, which was a workflow schedule, so this is a problem I tried to solve at Spotify was, we had a lot of different batch jobs, and you ended up with like very complex graph, you know, you need to run like hundreds of different jobs in a certain sequence, and sort of very, you know, parametric ways, and so I built Luigi to sort of, you know, help us figure that out, like how to execute all of it in like correct order. Uh, open sourced it. A bunch of people started using it. I think, you know, this is like two, 2011. Uh, then I like stopped really caring about in 2015, like Airflow kind of took over, and then like later, now there's like Daxter, and Prefect, and Flight, and a bunch of other ones. Um, I mean, I still think it's a good idea, and like kind of, you know, related to like how I ended up starting Modal, like part of, you know, how Modal came to be is I actually started looking again at workflow scheduling in late 2020 when I realized I wanted to start something or build something. And I started thinking a lot about like, you know, what makes a good workflow schedule. I even built one. Uh, I have some like janky code on my laptop. I realized at some point, like a workflow schedule is only as good as like the sort of underlying compute …

AI assessment note: “part of, you know, how Modal came to be is I actually started looking again”

Q You know, I know that you already decrease your price once to reflect the, a decrease in the price of GPUs. Uh, but how do you, How do you think about that layer of value between, ah, you know, what the GPU vendors or the cloud providers that provide the GPU, what they charge you and what you charge to customers?

A Yeah, I, I think, I think fundamentally, like through our software, we have the ability to charge more than the underlying GPU costs. Now, like how much more is the question, right? Like customers would probably freak out if we charge 10 X more because then they're just going to go, actually, you know what, like, you know, as much as I like the developer experience of modal, I'm not going to pay 10 X. I'm just going to go deploy it directly on like whatever Lambda Labs or something like that. And, and, and so the question is like, how much more can we charge? And, and, you know, it's like probably like two to three X, I think. You know, like, one of the things I always, like, think about is, like, are we a WeWork? Like, in the sense that, like, do we make, like, long-term GPU reservations, and then we have, like, short-term sort of, you know, income, and then, like, you know, we have this, like, massive sort of, you know, exposure to, to, to the underlying, you know, costs.

AI assessment note: “how much more can we charge? And, and, you know, it's like probably like two to three X”

Q You've had some interesting thoughts on how to manage the team in terms of giving them a lot of leeway. Uh, what, what is your philosophy on managing that early engineering team?

A I think that in the early days, actually like, you know, mid stage two, like you can get very far with almost no management and the sort of, you know, and I look at like a lot of like the lessons from like early Spotify days, like Spotify, you know, Spotify probably went too far. Like no one told me who my manager was for the last year, for the first year of Spotify. It was probably bad, but, but I think it's sort of an extreme example of like the sort of freedom under responsibility that existed in the early days of Spotify. Like at the end of the day, like people were super commercial and building the stuff that they thought was the best For the, for the company, uh, and kind of self-organize around that. There's a little bit of like scrum and stuff like that, but actually I think that was kind of bad. Um, so, so, so to me, like, you know, the sort of, like, I'm not naive about it, but I, but I think you're aspirationally, like you can get very close to like self-organization. If you just hire like fairly smart people, very entrepreneurial people, competitive people who, and, and, and just give them the context. You just like, tell them like, this is what we're trying to build, you know, and then you go figure it out. Uh, and then obviously like as you grow, then you're gonna have to like kind of put things in different swim lanes and have a little bit more like project manag…

AI assessment note: “you can get very close to like self-organization. If you just hire like fairly smart people”

Q In terms of the model itself, I just bring my own model. Effectively, I go on hugging face and grab some Open source model and deploy it on modal?

A Yeah, you can, you can definitely like take a hugging face model and deploy it. That's what a lot of people use modal for. I think where modal really shines is also like people training their own models, like having custom, like one, one, one example I always bring up is like, you know, amazing use case. I love the product is, is Suno, uh, which is AI generated music. And, and, you know, they, they have a big cluster. They train their own models outside of modal and then they use modal for the, the, the inference side. So, uh, which means, you know, like we run, you know, generate, like all this sort of generation of AI generated music happens on modal, like very large scale, right? Like we, you know, millions and millions of music generated on modal, um.

AI assessment note: “Yeah, you can, you can definitely like take a hugging face model and deploy it.”

Q And to finish the tour, uh, sandboxed code execution. What does that mean?

A Yeah. Like we, we started seeing a lot of customers like who build LMs, like, you know, needing, you know, especially like people like doing LMs for code, needing, uh, safe code execution. As it happens, like we, we, we had great primitives for that. Cause we all like, that's effectively what we built, right? Like we built like a system for like, you know, containing user code and executing in a safe way. So, so we started thinking about, can we expose that as a service? Uh, so we also have primitives to like take. Sort of untrusted code and execute that in a safe way. And I, I would like, you know, cut, put that in sort of category of like unproven, like it's still like sort of early days. We're trying to figure out exactly what the market opportunity looks like, but, but there's like so many startups like building, you know, LN for code, whether it's like, you know, code generation or migrations or, or things like that, or, or, or like other, like debugging or things like that. So to me, that's like, remains like an unproven, you know, but like potentially very large upside, like market opportunity for us. Uh, that I'm pretty excited about.

AI assessment note: “containing user code and executing in a safe way”

Q You mentioned batch processing, um, you know, job queues and all that stuff, which is, uh, you know, somewhere it feels more like the sort of more classic data engineering part. Is that, um, is that a big use case as well?

A Yeah. I mean, like, you know, like, like there's like a bunch of people using like us for like data parallels and like pipelines and stuff. I mean, there's like weird, Like sort of unexpected use cases. I shouldn't say weird. It sounds like I'm just missing that, but I actually, the fascinating use cases, like, you know, we've seen in like biotech, for instance, like, you know, companies having, I don't like, I don't know bio super well. So this might be like a gross generalized, you know, bad sort of characterization, but like scanning like millions and millions of compounds for like certain, you know, chemical properties or, or, or like medical imaging where, where customers have, You know, millions of, of computer, like running computer vision on like millions of, of, of things, you know, coming from medical imaging. So, so there is a lot of like sort of batch processing also in those use cases that I find fascinating and not, not sort of, not just like sort of data pipelines, which is what I think people think about with batch processing, but also all kinds of other, you know, interesting applications around, you know, biotech or, or video transcoding or feature extraction and things like that.

AI assessment note: “there is a lot of like sort of batch processing also in those use cases”

Q What do you think that is? What do you think you nailed to be able to get that amount of, uh, love and interest?

A I think we did a few things right, which was like, you know, one was just like me just, you know, building kind of the product I always wanted to have and just being kind of opinionated about like what that looks like, and then not being afraid to actually go deep into sort of layers of infrastructure and like change things that sort of got in the way of like delivering that experience. Um, to me, it's like we live and die by the quality of the, the ergonomics of the tool. And so to me, that's like the core competitive advantage. That's the sort of, you know, core ethos of the company. Like we cannot absolutely lose that. Uh, that's why people pick modal. Everything else is sort of, you know, secondary. I also have a long background in building consumer products my whole career. So I think that's maybe helped, uh, kind of thinking about, you know, what is the onboarding experience? Like, what does it feel like? It's sort of unboxing, like, you know, when you get started, you, you know, download a client, like install it and like, you know, like, what does that feel like? I think that's like kind of often like a missed opportunity to like, create the sort of set the expectation that this is a magic tool. Uh, so spending a lot of time thinking about that, I think, you know, it was also sort of inherently led to more like a, you know, bottoms up sort of, you know, go to market app…

AI assessment note: “we live and die by the quality of the, the ergonomics of the tool.”

Q And how does that work on a daily basis? Uh, cause you sort of, uh, don't manage them, but, you know, ultimately manage them at the same time by virtue of being the, the CEO is like, do you like catch up or they come to you when they have a, uh, a problem or you just look at the final product? Like, how does that work?

A Yeah. I mean, I think you can look at, you know, the final product, you can talk about what they're working on. Like, I, I, I don't, you know, necessarily believe in those sort of estimates or, or, uh, you know, uh, deadlines or things like that, but, but I do think there's an element of like looking across the portfolio of your current project and like continuously sort of adjusting. Are we spending too much time on this? Are we spending not enough time on this other thing? And sort of continuously adjusting that every month we do like a lightweight sort of, you know, set the priorities for the month and like, here are the things we're trying to accomplish. Uh, communicate with, with all teams, you know, make sure they're, you know, there's nothing that's missing, sort of the missy, like, you know, you have everything covered and there's nothing that's missing. Um, and, um, but, but that's like a notion doc. That's like super, very basic. And then, you know, we, we do track things in linear, like, you know, but, but, but for, for most of the sort of day-to-day management, it's like people just come in and write a lot of code.

AI assessment note: “for most of the sort of day-to-day management, it's like people just come in”

Q The spectrum between self-hosted or on-prem to cloud maximalism is something we think about a lot as well. I mean, there does seem to be a somewhat surprising pull towards on-prem these days or self-hosted. Why do you think that is?

A I think it's just easier to grow your business that way, right? Like, you know, a lot of companies like, you know, first of all, like for a lot of, you know, for a lot of products, like it's not like necessarily like a core You know, impact on revenue, right? Like if you're building a SaaS business and, you know, self-hosting versus, you know, multi-tenancy, like if you fundamentally charge roughly the same thing, or maybe even more for self-hosting, I don't know, like then it doesn't really, you know, make big difference. And, and if the quality of the service is kind of similar, then like, you know, why even like, you know, bother like insisting, right? Like, let's just like go and meet the customers where their demand is. Uh, for us, I think the product is much better With the multi-tenant service and the revenue potential is much larger because we have full control of the underlying COGS, like the underlying costs, and we can do all these optimizations. So for us, I think there's more of a case to be made for like pushing for this thing, uh, for, for, for many other types of business, I don't think it matters that much. And, and so I think they're better off just like, you know, meeting the customers where they are. Uh, but I do think there's a lot of customers also, a lot of vendors out there that like, maybe they should push more for like, you know, sort of cloud maximali…

AI assessment note: “I think it's just easier to grow your business that way”

Q You know, that, that moment is always very interesting in the life of a company, right? So the, the salesperson, what, what do they do? Do they, uh, take some of the inbound and just like have that first conversation? What, what, uh, and I, I know you're figuring it out, but like, what do they do currently?

A Yeah, I mean, there's a fair bit of that. I also think fundamentally, like, you know, when you look at like later stage companies, like I, I do think that a more traditional sales model will make sense. You know, for those companies, like if you're selling to, I don't know, I mean like in extreme case with you like Bank of America, like you, you can't expect them to like, you know, sign up using GitHub and just like start writing code and deploying things and swipe a credit card, right? Like there's going to be some procurement process. There's going to be, you know, some proof of concept. There's going to be a bunch of other stuff. So we're trying to figure out exactly how that looks like and how it fits into our, our model. Um, we've been very like happy, you know, with the inbound and just like kind of upselling and like, that's been, you know, Enough to get to the point where we are, but I also think like layering also sort of top down, you know, will, will help us get to the next layer. I look at, I mean, to me, like actually like the best companies and, and, you know, that, that I look at for inspiration would be like, you know, Datadog or Mongo or something like that, like started with, or AWS actually, frankly, like started with a, with a, with a product that customers loved and nailed the sort of, you know, bottoms up, like, you know, self-service process. But then als…

AI assessment note: “Yeah, I mean, there's a fair bit of that.”