data

“I think the difference is that like for reinforcement fine tuning, you're collecting data that's going to be useful As the models improve as well. So if we come out with, like, future models that are even more capable, you could still fine tune them on your da…”

“Something I've said before and I'll say again is, is that data is the most under-invested in area of research relative to its impact, and I don't think it's even close.”

“If you talk to the most talented AI researchers and you ask them, what's the secret to your success, they'll largely tell you that they look at the data.”

“We should be thinking about spending as much on the environments as on the compute and the data, because the environments literally shape the, what intelligence can emerge.”