LLM Routing
topic on 1 show · 1 statements across 1 episodes
1 statements about LLM Routing, every show
Beauchamp: Dynamic LLM routing is expensive and offers minimal performance edge
“You can try and do, like, a routing thing where you say, for a user, given user requests, we're going to try and predict which of these end models that users enjoy the most. That turns out to be pretty expensive and not a huge source of, like, edge or improvem…”