The Exchanges

Q Yeah. I mean, what you describe is a pretty broad platform based approach. Um, I think there's a handful of companies who are sort of in your general space or market. How do you feel that modal differentiates from them?

A I think, first of all, we, we're cloud native. Like, we're just like cloud maximalists. Like, we went all in and said, like, wait a second, we're going to build a multi-tenant platform that runs everyone's computer. And the benefits of that are, like, very tremendous, because, like, we could just do capacity management much better, and that's one of the ways we can offer, like, instantaneous access to hundreds of GPUs if you need to. Like, you can do these, like, very bursty things, and we just give you lots of GPUs, right? I, I think the other benefit, or the other sort of differentiation is to be very general purpose. Uh, we focus on sort of what I think, as I mentioned, like high code, like we run custom code in our containers, in our infrastructure, which is a harder problem. Like containerization and running user code in a safe way is a hard problem. And then dealing with container cold start. And like I mentioned, we have to build our own scheduler. We have to build our own container runtime in our own file system to boot containers very quickly. Uh, and, and I think so unlike many other vendors, like they're only focused on say inference or maybe only LMs. Uh, our, our approach has always been to build a very general purpose platform and, and, and sort of, you know, in the long run, I, I hope to sort of that, that, that sort of manifestation will be more clear because I …

AI assessment note: “the other sort of differentiation is to be very general purpose.”

Q Um, I, if I remember correctly, you were, you were an IOI gold medalist. Yeah, that's right. And obviously you think a lot about code and coding, and how do you think that changes with AI over time? Or do you have any contrarian predictions on, on what happens there?

A I don't know if this is contrarian, but like, I actually think that like, you know, this is just like one out of many improvements in developer productivity. And, you know, you look back at like, you know, whatever, like compilers was originally like, you know, a tool that made developers more productive and then like higher level programming languages and databases and cloud and all these things. And so like, I actually don't know if like AI is like, you know, different than any of those changes in the hindsight. And so, and, and, and by the way, like every time that's happened, you know, It turns out like there's so much latent demand for software that actually like the number of software engineers goes up. So like, I feel like you look back at like, you know, last like four years of software development, like every decade engineers get like 10 times more productive due to better frameworks or better, you know, tooling or whatever. And, and it turns out actually that just unlocks more latent demand for software engineers. So I, I'm very bullish on software engineers. I think it would take a lot to sort of destroy that demand. I think people look at a lot of like AI as like a kind of fix something, but, but in my opinion, it's like, No, it's just gonna unlock more latent demand for more things. So I'm very bullish on software engineering.

AI assessment note: “this is just like one out of many improvements in developer productivity”

Q What else do you think is missing in the world today in terms of AI infrastructure or infrastructure as a service?

A I mean, I'm very biased, but I think modal is like basically a way to like for, for engineers to, to take code and run it. And look, I'm very bullish on like, you know, code and like people wanting to write code and building stuff themselves. I think outside of sort of LM space, which is like a very kind of different world, in my opinion, I think there's always going to be a lot of applications where people want to train their own models. They want to run their own models or, or at least like run other models, but have like very custom workflows. Uh, and, and I just don't think there's been a great way to do that. It's like, Pretty painful to do that. And so I, I think that's pretty exciting. I think on the storage side, there's some other really exciting stuff. Like we, we haven't really touched storage at modal. Like we, we focus very much on compute. So I'm personally very interested in sort of vector database. Like how's that going to evolve? I don't think anyone really knows. Um, I'm pretty interested in like, you know, more efficient storage around training data. I'm also very interested in like, I guess another thing I'm, I'm very fascinated by right now is, um, Uh, training workloads. Uh, in order to, to train large models efficiently, you have to really spend a lot of money and time setting up the networking. So one of the things I'm really excited about is what if you…

AI assessment note: “I think on the storage side, there's some other really exciting stuff.”

Q It's super interesting. Could you say more on that?

A I mean, like one thing I think a lot about is like maybe the database itself be like the embedding engine, right? Like instead of like you put a vector in a, you know, you search by that vector, I think there's a lot of, you know, The more native, like AI native storage solution would be, you put text in, you put, you know, video in, you put image in, and then you can search by that. Like to me, that would be like a more sort of native, AI native sort of storage solution. So that's like one line of thought that I've had is like, maybe we just, we're just like so early to this that like, I think it's going to take five, 10 years for it to really For, for it to shake out.

AI assessment note: “The more native, like AI native storage solution would be, you put text in”

Q about things like, um, latency or pings out to other third party services versus just running on their own existing cloud provider or their hyperscaler that they work with or set of hyperscalers. You know, many of them actually, uh, work across multiple. How do you think about that in the context of modal in terms of your own compute versus hyperscalers versus, you know, the ability to run anywhere?

A Yeah, totally. And, and of course, there's also a sort of security compliance aspect of this. Like, I, I, I think, you know, it is a, it is a, you know, challenge. Uh, I, I look back at when the cloud came and I remember back in like, 2008, 2009 and the cloud came and my first reaction was like, how the hell would like, why, why would anyone put their computer in someone else's computer and like run that? And, and I think, you know, to me, that was just like insane. Like, why would anyone do that? But over the next couple of years, I was like, actually kind of makes a lot of sense. And, and I think now even like among like enterprise companies, It's like, there's a sort of recognition that like, yeah, actually probably our computer is more safe in the big hyperscalers. And in a similar, similar vein, I remember talking to Snowflake back in say, 2012 or something like that. And they had a sort of similar approach where like, they basically said like, we're going to run databases in the cloud and it's not going to be in your environment, you know, or maybe in your environment, but like we're in infrastructure as a service. And I thought that was nuts. And then obviously like, I think Snowflake now is a very large, you know, publicly traded company. I think they showed that like, Infrastructure as a service makes a lot of sense. And so I, I think there is a little bit of resistanc…

AI assessment note: “And of course, there's also a sort of security compliance aspect of this.”

Q the first European technology companies to get there, um, which is pretty cool. So a lot of folks I know, um, may use one of the existing vector DBs, or in some cases are just using Postgres with, um, with PG vector, right? How do you think about the need for vector databases as sort of standalone Pieces of infrastructure versus just, you know, adopting Postgres versus doing something else.

A Yeah, I feel like everyone's debating that. I, I don't know necessarily. Like, I, I think there's a lot of, there's a case to be made that, you know, you can just stick everything into relational database and, and you're, you're fine. To me, like the, the bigger question is like in the long run, like, you know, if you think about like, what's like an AI native data storage solution, like, I don't even know if it's like necessarily has the same form factors and the same interface as, as a database. So that's actually a bigger question that I'm more excited about is like, I think people look at like vector databases and like, You know, whether it's relational or not, they sort of shoehorn it into this, like, you know, sort of old school model of like you, you put data, you get data back, but I don't know. I, I think there's like a lot of room to sort of rethink that in the age of AI and have very different, like, you know, interaction models with that data. I know that sounds a little fluffy.

AI assessment note: “that's actually a bigger question that I'm more excited about”