Direct Preference Optimization
also referred to as: dpo
8 statements across 4 episodes · 4 bullish · 1 bearish · 4 people on the record · first statement Jan 11, 2024 by Nathan Lambert · said 4 times in 3 episodes since 2024 · across every show →
Mentions by year
brought up most by Soumith Chintala (2)
tap a year for its mentions
2025 1 mention in 1 episode
Everything said about Direct Preference Optimization, oldest first
Jan 11, 2024 neutral
Lambert: AI2 trained 70B TÜLU 2 on the first run without ablations
“Let's just try the Zephyr recipe on seventy billion parameters, and it's literally, like, the first run. It's like, we did no ablations, didn't change any parameters, we just copied them all over. And like, that's the model that people have been working with”
Jan 11, 2024
Jan 11, 2024 positive
Lambert: DPO Has Become the Standard Release Expectation for Open-Source LLMs
“I think DPO releases are kind of becoming expected because Mistral released a DPO model as well. I think the slide after this is just like, there's a ton. It's like Intel releases DPO models, Stability releases DPO models. At some point, you just have to accep…”
Jan 11, 2024 bullish
Jul 5, 2024 negative
Yi Tay: Distilled open-source model variants disappeared after failing to climb LMSYS
“When people realize that, like, this, like, turning on the GPT-IV tab and running some DPO is not going to give them the reward signal that they want anymore, right? Then all these variants gone, right? You know, there was this era where there's, wow, there's …”
Dec 23, 2024 positive
Soldani: Ai2 Trains OLMo Using Community Datasets and Other Models' Outputs
“We see a lot of these even in our own work of like, you know, as we iterate in the various version of Olmo it's not just like every time we collect from scratch all the data. No, the first step is like, okay, what are the cool data sources and datasets people …”
Mar 30, 2026 positive
Lample: Post-training breakthroughs like DPO were impossible without open LLaMA
“And if you look at many of the techniques that were developed after, for instance, Temma was open source, like all these post-training approaches like even DPOD, like performance optimization, all of this were done by people that had access to this model, and …”