Pre Training Paradigm

“That is separate from the new reasoning paradigm that has emerged in the past year which is where you train models to on math and coding types of questions with reinforcement learning, and that has a very steep slope, and I don't see any signs of that slowing …”

“That sort of paradigm does seem like it's running out of steam. It has held for many, many orders of magnitude but the returns on doing that are lower.”