reinforcement learning

“Actually, DeepMind are another example of the same thing, because if you want to apply reinforcement learning to games, and that's enabled them to make some quite fundamental sort of progress, you don't need vast amounts of data either.”

“According to the Alan Soufre AI Semantics Scholar, he's the most highly cited researcher in reinforcement learning, 11th most influential researcher in all of competing science. And his textbook on reinforcement learning was ranked as the single most influenti…”

“Right, where there's so much momentum right now that says, basically, we're one labeled data set away from glory. Right, and this result basically shows you, wow, there's a lot of mileage that you can get out of reinforcement learning where there's no data, no…”

“So if we do our reinforcement learning and we skew it towards code snippets that's generating that don't have API keys, inadvertently, we may be training this thing to behave less like a data scientist. And then we lose the entire discipline of data science in…”

“And I think the big thing in, in R-one, or generally with these reasoning models is, We were doing before there was a human in the loop always, right? Like when we have this SFT training and these other techniques that we're doing after like RLHF and having R …”

“And supervised learning, you can only do as well as the best human that you have. Because what's happening is that you're learning from the teacher. In reinforcement learning, you're learning from the world. You're able to learn things better than the best hum…”

“We're at this point where kind of like the compute and machine learning, reinforcement learning Do really enable, enable you to go no humans in the loop and how a lot of these plants are controlled.”

“As you like build more of that large scale infrastructure and kind of like see the problems that humans have to solve on a daily basis, it becomes like pretty obvious that these are problems that humans don't, aren't best positioned to solve. Like these are la…”

“But as soon as you're doing RL, where you're training it in a specific domain with a specific verifier, you're likely losing, you know, other areas. So you make it really, really good at playing chess. It's going to be less good at, you know, something else li…”

“Training a model to be good at a specific capability is very data efficient. You don't need that many examples to teach it something new.”

“When we saw the reinforcement learning algorithm working really well on math and physics problems and coding problems, It became pretty clear, like, just from reading through the chain of thought, like, okay, this thing's actually, like, thinking and reasoning…”

“I expect this will evolve quite rapidly. I expect it will become simpler, right? Like I think, you know, maybe like two years ago, we would have been talking about like, what is the right way to craft my fine tuning data set? And I don't think we are like at t…”

“High compute reinforcement learning is really effective. This is how you should think about the strategies it's using. This is how you create effective tool using towards those problems, and this is how you optimize it effectively.”

“All these techniques that have been developed over the last few years, Seems to me they're absolutely gonna apply to a problem like a humanoid robot as well.”

“I think it's RL. I think it's reinforcement learning.”

“I think the breakthroughs in RL are incredibly exciting, but we also knew about them now for like over 10 years where you marry generative systems with TreeSearch and things like that. But there's a lot more to go there”

“Foundation model companies are, in some cases, they are hiring, they're actually hiring human experts. To generate new training data. So they're actually hiring mathematicians and physicists and coders to basically sit, and, you know, they're hiring human prog…”

“And very quickly they stopped working on language models, and we were very upset because we really wanted to continue going there, but they didn't want to talk about any language models because no one was really working on them and that may have made us feel v…”

“Care is basically like reward. Like how much does this state correlate with survival? How much does this state correlate with your inclusive, your full inclusive reproductive fitness? For a somewhat thing that learns evolutionarily, or for a reinforcement lear…”

“We're making a big bet on autonomy and refineries, where we use reinforcement learning to actually remove humans from the loop in determining how refineries operate.”

“If you actually have a problem that you can specialize the model around with your reasoning traces, You can start to create this compounding advantage in your domain for your customer base, where you're able to kind of shape the intelligence to be better than …”

“I think what is definitely true is that, like, the skill of, like, developing a theory or, like, building your understanding of some poorly understood object is, like, a fuzzier one. So it might be harder, you know I guess you can try, you can tell it, you kno…”