The Exchanges

Q We were kind of talking about it before, and there's this weird thing where One week is more expensive of both one day and one month. What are like some of the market pricing dynamics? What are things that like this to somebody that is not in the business? This looks really weird, but I'm curious, like if you have an explanation for it, if that looks normal to you.

A Yeah. So the, the simple answer is preemptible pricing is cheaper than non-preemptible pricing. And the same economic principle is the reason why that's the case right now. That's not entirely true on SF Compute. SF Compute doesn't really have the concept of preemptible. Instead, what it has is very short reservations. So, you know, you go to a traditional cloud provider and you can say, hey, I want to reserve contract for a year. We will let you do a reserve contract for one hour, which is the part of SFC. Um, but what you can do is you can just buy every single hour continuously, um, and you're reserving just for that hour. And then the next hour you reserve just for that next hour. And this is obviously like a built-in, this is like an automation that you can use. But what you're seeing when you see the cheap price is you're seeing somebody who's buying the next hour, but maybe not necessarily buying an hour after that. So if the price goes Up too much. They might not get that next hour. And the underlying part of this, of where that's coming from in the market, is you can imagine like day-old milk, or like milk that's about to be old, it might drop its price until it's expired, um, because nobody wants to buy the milk that's in the past, or maybe you can't legally sell it. Compute is the same way. No, you can't sell a block of compute that is not, that is in the past. And s…

AI assessment note: “preemptible pricing is cheaper than non-preemptible pricing”

Q What are the utilization rates at which a market like this works? What do you see the usual GBU utilization rate and like at what point? Does the market get saturated?

A Assuming there are not, like, hardware problems or software problems, the utilization rate is, like, near a hundred percent, because the price dips until the utilization is a hundred percent. So the price actually has to dip quite a lot in order for the utilization not to be. That's not always the case because you just have logistical problems, um, like, you get a cluster and parts of the InfiniBand fabric are broken, and there's, like, um, some issue with some switch somewhere, and so you have to take some portion of the cluster offline, or, you know, stuff like this. Like there's just underlying physical realities of the clusters, but nominally, um, we have better utilization than basically anybody because, um, but that's on utilization of the cluster. Like that doesn't necessarily translate into, um, well, I mean, I actually do think we have much better overall money made for our underlying vendors than kind of anybody else. We work with the other GPU clouds, um, and the basic pitch to the other GPU clouds is one, we're still your broker. So we can, we can find you the long-term contracts that are at the prices that you want, but Meanwhile, your cluster is idle. And for that, we can increase your utilization and get you more money because we can sell that idle cluster for you. And then the moment we find the longer, the bigger customer and they come on, you can kick off thos…

AI assessment note: “the utilization rate is, like, near a hundred percent, because the price dips”

Q Um, yeah, so you kind of talked about the different providers. Why did you decide to go with this approach and maybe talk a bit about how the market dynamics have evolved since you started the company?

A So originally we were not doing this at all. Um, we were definitely, like, forced into this to some extent. SF Compute started because, uh, we wanted to go train models. For music, um, and audio in general, we, we were going to do a sort of generic audio model and at some points, and then we were going to do a music model at some points. It was early company. We didn't really spec down on a particular thing, but yeah, we were going to do a music model and audio model. First thing that you do when you start any AI lab is you go out and you buy a big cluster. The thing we had seen everybody else do was they went out and they raised a really big round and then they would get stuck. Um, because if you raise the amount of money that you need to train a model initially, Like, you know, the fifty million dollar pre-seed pre-revenue, um, your valuation is so high, or you get diluted so much, um, that you can't raise the next round. Um, and that's a very big ask to make. And also, I don't know, I, I felt like we just felt like we couldn't do it. We probably could have in retrospect, but, um, I think one, we didn't really feel like we could do it. Two, it felt like if we did, we would have been stuck later on. We didn't want to raise the big round. And so instead we thought surely by now, um, we would be able to Just go out to any provider and buy, like, a traditional CPU cloud would sel…

AI assessment note: “originally we were not doing this at all. Um, we were definitely, like, forced into this”

Q Any customers or, like, stories that you want to shout out of, like, maybe things that wouldn't have been economically viable, like others? I know there's some sensitivity on, on that, but.

A My, my favorites are grad students, are folks who are trying to do things that would normally otherwise require the scale of a big lab. And the grad students are like the worst possible customer for the traditional GPU clouds, because they will immediately turn, um, if you sell them a thing, because they're going to graduate and then like, I'm going to go anywhere or they're not, they're not going to like, that project isn't continuing to spend lots of money. Like sometimes it does, but not, um, if you're like working with the university or you're working with the lab, I'm sorry, but a lot of times it's just like the ability for us to offer like big burst capacity, I think is lovely and wonderful. And it's like one of my favorite things to do because All those folks look like we did, um, and I have a special place in my heart for young hackers and young grad students and researchers who are trying to do the same genre of thing that we are doing. For the same reason, I have a special place in my heart for, like, the startups, um, the people who are just actively trying to compete on the same scale, um, but can't afford it time-wise, but can't afford it, um, you know, spike-wise.

AI assessment note: “My, my favorites are grad students, are folks who are trying to do things”