Compute Scaling

“I think there are still major, like, compute multipliers, major ways of saving compute that can lead to better performance without just naively scaling.”

“When we are thinking about our architectures, Like the architectural elements, the algorithms that we put in there that come, that make up the model, right? They are as important as the scale.”

“This wasn't just a matter of scaling compute. It really was an algorithmic breakthrough, and this kind of result would have been doable 20 years ago if people knew the approach to dig.”