Reinforcement Learning Infrastructure
topic on 1 show · 1 statements across 1 episodes
1 statements about Reinforcement Learning Infrastructure, every show
Lambert: Long inference generations break RL infrastructure and require more GPUs
“The inference, high inference length generations definitely just, like, kind of breaks all infrastructure, because there's just so many tokens, there's more opportunity for out of memory or other things to go wrong. So it's like, just on a default, all of your…”