AI Model Serving
topic on 1 show · 1 statements across 1 episodes
1 statements about AI Model Serving, every show
Shazeer: Centralized AI Inference Batching Is Up to 100x More Efficient
“Obviously though, there's also like economics of scale of both training the best models And serving, like if you want to serve a product, you can do it maybe a hundred times more efficiently if you're serving many, many people at once and kind of batching thin…”