Asynchronous Training Algorithms
topic on 1 show · 1 statements across 1 episodes
1 statements about Asynchronous Training Algorithms, every show
Peng: Starting distributed AI optimization with fully asynchronous nodes sacrifices convergence
“A lot of optimization algorithms start, like, with everything being, like, asynchronous. Right. But that's, like, really hard to get because you sacrifice a lot. You sacrifice the speed. You sacrifice the convergence, right? Like, actual efficiency of the algo…”