compute

“What matters in reality is making things work. And it's often the simplest ideas that work in practice, especially when thrown a lot of compute at them. The simplest ideas typically outshine the complicated ones.”

“Compute alone is useless. Like, people have tried to reproduce these things with doing the same thing, and it doesn't work. You gotta throw high-quality data tokens at the problem, too.”

“If you scale up models, you give them more data, more compute. Again, there are a few modifications like RL, but not really very much. It's pretty close to pure scaling. You find that, you know, when you do that, you find, you know, incredible increases in per…”