inference

“Our prediction is for those kind of applications, the inference is much more important than training. Because inference scale is proportional to the upliminal world population. And training. Training scale is proportional to the number of researchers.”

“Our thesis is inference is not the business. We, yeah, we want to give the end user total transparency into price, into, which I think is like incredibly important to, you know, even get comfortable with the idea of spending as much money as you do. I think th…”

“When you think about the total cost of ownership of these models, it's gonna be very, very heavily weighted towards inference. It's all inference.”

“If you must update weights, then, like, you know, weights become stateful, and today, like, inference is not stateful.”

“Primarily inference that funds the free usage of the platform and then also hiring, of course, headcount. We have an office, you know. That's an SF.”

“And with training, it's more of like a math, like you can run the math and see the flops and maximize it. With inference, it's more of like an auto-tuning, like GPU kernel auto-tuning... You shadow the same traffic, like real traffic, and you just see which co…”

“Like, you'll, you'll know that influence is pretty much solved when researchers start publishing about how they got one percent faster at something.”