reinforcement learning

“You know, I think the at a high level, you know, I think all the continued progress that we're seeing everywhere in AI actually is, is primarily due to Essentially like reinforcement learning, RL, that's really working.”

“My broader take on the human data market is that It's going to grow dramatically because, so we're getting to the point where RL is so effective that you can create almost any eval and it will be able to solve that eval. And so the barrier to applying AI throu…”

“But I think if you look at like a, an example of this in practice, like 2.5 pro is actually an example where it wasn't just like RL scaling that made that model better. Yes, RL was part of the story, but like there was also a bunch of pre-training innovation a…”

“And this is why, like, I don't subscribe to the, like pre-training is, you know, dead and all that stuff, because the more work that you can do at the pre-training level, those capabilities, as you do post-training and as you give the models RL capability, it'…”

“I think RL is the place where most investments are gonna go to especially with models like O-three that are able to do tool calls pretty natively rather than being prompt engineered to do that”

“The reason O-three is good is because it's trained to use tools. The way you train a model to use the right tool for the job is reinforcement learning. And they've said as much, like, deep research, reinforcement learning.”

“We're still seeing these huge gains where you go, you know, 10 X compute increase in RL. We still getting like very distinct linear gains for that.”

“I think, like, if you find the right levers, you can really pump a lot of compute into RL as well as pre-training.”

“We've got very little rewards right now, but pretty quickly over the next year or two, you're going to start to see much more meaningful and long horizon rewards.”

“A lot of the reinforcement learning is now done on like extremely hard math problems. And so you might need to go to PhD programs to recruit.”

“Because like tokens, you kind of hit diminishing returns pretty quick in terms of just like more internet texts or more like human written math solution examples. And it doesn't scale super well, but the thing, the nice thing about the task specification is yo…”

“I think the bigger problem is not just the width or the width of the pool, how many different tasks you have to RL on, but it's a depth in the sense that a job doesn't involve doing a thousand different five minute tasks individually. It's the fact that you're…”

“There's a ton of situations where you can go and RL on a specific task and it gets really, really good at it, but then you try and get it to do anything else and it's not that great.”

“Pre-training is a more efficient form of training. Because you can advertise it.”

“Long context reinforcement learning has been very, very successful in the coding world because Git has a complete history of every line of code that's been written, every comment, why it happened.”