PPO
topic on 4 shows · 7 statements across 6 episodes · said 1 times in 1 episodes since 2025
Latent Space 1 
the MAD Podcast 
All-In 
TBPN
Mentions by year, every show
tap a year for its mentions
Latent Space 1
every mention on every show, scene by scene, with the transcript →
7 statements about PPO, every show
Eysenbach: 1,000-layer RL requires reward-free objectives, not just architectural tricks
“I think the main conclusion is that using big networks not only requires these architectural tricks, but also, as Kevin mentioned before, it requires using a different objective. This objective doesn't actually use rewards in it, and so there's another word in…”
Lambert: Most AI labs probably use evolved GRPO rather than PPO
“In reality, it seems like most people are using something like an evolved version of GRPO, which is a bit simpler than PPO.”
Corbitt: PPO enables training purely on real production traces without simulation
“And PPO, now in practice, a lot of times when you're training with PPO, you also will use an environment like that because it lets you do a bunch of runs and be more data efficient. But at least in principle, you have the option with PPO, you can actually, lik…”
Srinivas: Perplexity plans to invest more in RL post-training this year
“The nice thing is a lot of open source code bases exist on how to replicate GRPO or PPO and post training these models. And we've been doing that work already. So that's where we plan to invest more resources into for this year.”
Palihapitiya: DeepSeek created GRPO algorithm to slash AI memory requirements
“They invented a totally different algorithm. There was the orthodoxy. Right? This thing called PPO that everybody used, and they were like, no, we're going to use something else called, I think it's called GRPO or something. It uses a lot less computer memory,…”
Lambert: Meta Used Rejection Sampling to Bootstrapping Llama 2 RLHF
“Llama started their RLHF process with this to get some signal out of preference data. That preference data went into a reward model, and then the reward model did a good enough ranking that it was, like, essentially superpowered instruction tuning based on rew…”