data curation

“The easiest way to think about this is that the value of a data point is not just a function of that data point itself. It's rather a function of how that data point relates to every other data point in the training set, right?”

“Making the data better can be a massive compute multiplier. It can change the performance per dollar by orders of magnitude.”

“Data creation also is a hard problem to solve quote unquote, because it's not one where there's a single silver bullet. There's not just do this one trick and all of a sudden things work. It's rather here are these 50 different things that you can do, each of …”

“You know, we've already been able to get 10 X gains. I think there's at least another hundred X behind this that are still to be done.”

“And what that paper showed was that if you use your data correctly, you can actually bend the scaling laws themselves.”