Model Routing

“Really the hardest part of routing is building the evals and trying to determine in what places a set of intelligences should be used for a particular application.”

“One of the problems when you do model routing is there's some companies like Anthropix spends a lot of time. I know we make fun of them online, but they spend a lot of time actually making sure all of their model families roughly don't change too much in terms…”

“The way I think about our stack is there is the part of the stack that deals with the user interface, like the feel and the personality. And there it is harder for me to just route Wildly, because I need consistency of the experience that is sometimes hard to …”

“I think right now, more and more of that in, in, in, in, in this recent trend, we're going to probably see also, also more of this coming up when it's not going to be coming down to cost only, but also to the performance of those models and making sure that a …”

“In fact, I think the Apple, so I can tell you model routing has become like so dominant in conversations in my world now. Like it went from, you know, like just there's, you have to have the latest frontier model and they're all encompassing. And within the sp…”

“For us, we've actually like figured that the like unit, the like The return on it was not actually that beneficial. What makes sense for us instead is, you know, like we use base models for like the final, like response. What makes sense, what makes more sense…”

“And so I pick and OpenAI have both keys that model routing on APIs already. And so I think they'll launch them, especially at some point where you can sort of prioritize the three tradeoffs that are in model routing, cost, speed, intelligence.”

“For the most part, designing these systems where you're kind of routing to different levels of intelligence is a really time-consuming and difficult task, and, like, it's probably worth it to just use the smart model And pay for it at this point if you're look…”