open source models

“Clem, CEO at Hugging Face, a few days ago, forecasted that in 2024, open source models are going to reach the level of the best closed source models. That may be aggressive for 20, 24, but they are headed towards parity.”

“But I really think that will converge to a setting where you have proprietary models and the open source models are just as good.”

“The irony right now in the market is that is that the best open source models are Chinese.”

“If we ran our entire business exclusively on frontier models, I would care a lot about costs, and I would think a lot about that, but once you're already, once you've already made the jump to saying, okay, now, now we know how to think about open source models…”

“I think they'll get there, but it'll probably take longer than people think because, you know, even with our team, fine tuning these models is non-trivial.”

“When we have open source open source models that we want to fine tune, we find that if we can clearly tailor our evals to customer outcomes, it's way better than just looking at, like, loss curves over time, right?”

“We don't just build our set of open source models and then, you know, it's done, we can move on to our next thing, and maybe we'll revisit this in two years. You often need to train new models all the time because As the frontier changes, as the capability of …”

“But I think the point is that at a certain point, it's strictly better to use open source models because when your use case is solidified and you're in production at scale and you're pretty sure this is the sort of shape of the agent, then there's no reason no…”

“At any given point in time, including now, I think models from open AI and Anthropic are kind of more widely used and more critical kind of in general than open source models.”