AI model serving
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Aug 31, 2023 by Noam Shazeer · across every show →
Everything said about AI model serving, oldest first
Aug 31, 2023
Shazeer: Centralized AI Inference Batching Is Up to 100x More Efficient
“Obviously though, there's also like economics of scale of both training the best models And serving, like if you want to serve a product, you can do it maybe a hundred times more efficiently if you're serving many, many people at once and kind of batching thin…”