Asynchronous Model Merging
topic on 1 show · 1 statements across 1 episodes
1 statements about Asynchronous Model Merging, every show
Weight updates across specialized tasks are orthogonal enough to merge asynchronously
“The updates made to model weights are orthogonal enough for specialized tasks that this is actually like totally fine. Things are nice and linear in most cases, things are nice and orthogonal, and you can get away with a lot of async updates to models that are…”