Multi Node Inference
topic on 1 show · 1 statements across 1 episodes
1 statements about Multi Node Inference, every show
Pipeline parallelism is only necessary for multi-node AI model inference
“The only reason you would have to do pipeline parallelism which is where you separate, like, different layers, and you put, like, half the layers on one hardware and half on another, is if you are forced to do multi-node inference because a model is bigger tha…”