asynchronous training algorithms
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Oct 1, 2024 by Bowen Peng · across every show →
Everything said about asynchronous training algorithms, oldest first
Oct 1, 2024
Peng: Starting distributed AI optimization with fully asynchronous nodes sacrifices convergence
“A lot of optimization algorithms start, like, with everything being, like, asynchronous. Right. But that's, like, really hard to get because you sacrifice a lot. You sacrifice the speed. You sacrifice the convergence, right? Like, actual efficiency of the algo…”