Data Curation
topic on 1 show · 5 statements across 1 episodes
5 statements about Data Curation, every show
Morcos: Data curation still has at least 100x in performance gains ahead
“You know, we've already been able to get 10 X gains. I think there's at least another hundred X behind this that are still to be done.”
Morcos: Proper data curation can bend neural scaling laws
“And what that paper showed was that if you use your data correctly, you can actually bend the scaling laws themselves.”
Morcos: A data point's value depends on its relationship to the full dataset
“The easiest way to think about this is that the value of a data point is not just a function of that data point itself. It's rather a function of how that data point relates to every other data point in the training set, right?”
Morcos: Data curation requires compounding dozens of individually modest, conflicting techniques
“Data creation also is a hard problem to solve quote unquote, because it's not one where there's a single silver bullet. There's not just do this one trick and all of a sudden things work. It's rather here are these 50 different things that you can do, each of …”