InstructGPT
2 statements across 2 episodes · 0 bullish · 0 bearish · 2 people on the record · first statement Nov 3, 2023 by Michael Royzen · said 13 times in 7 episodes since 2023 · across every show →
Mentions by year
brought up most by Nathan Lambert (4), Jack Morris (3), Thomas Scialom (2), Will Brown (1), Michael Royzen (1), Jason Liu (1)
tap a year for its mentions
2025 5 mentions in 3 episodes 2 per episode
2024 7 mentions in 3 episodes 2 per episode
2023 1 mention in 1 episode
Everything said about InstructGPT, oldest first
Nov 3, 2023
Royzen: BigScience's T-Zero predated InstructGPT in large-scale instruction tuning
“I think T-Zero is the first model that did large-scale instruction tuning from diverse data sources in the fall of twenty-twenty-one. This is before InstructGPT. This is before Flan T-Five, which came out in twenty-twenty-two. This is, I think, the very, very …”
Jan 11, 2024 neutral
Lambert: RLHF Performance Depends on Data and Systems Over RL Details
“It really ends up being kind of, like, gibberish that I think is less important now, because it's more about data and infrastructure than RL details, than, like, value functions and everything. A lot of the papers have different terms in the equations. I think…”