Ad Hoc GPUs
topic on 1 show · 1 statements across 1 episodes
1 statements about Ad Hoc GPUs, every show
Tamir: Serving 70B open-source LLMs forces teams to buy reserved GPUs
“What if you want to do something like a seventy billion Lama or minstrel or mixed or any of these, those are going to be very hard to get on an ad hoc basis. And you're going to end up having to pay the price for a reserve instance, if you want to be able to s…”