synthetic data

“So a project we're working on this year is to create what we call synthetic data, it's just a fancy word for simulated data. What we want to do is create a simulated data set That looks very much like the data that we have, but isn't.”

“At some point, like we lost saturated on data that, that exists on the internet. That's interesting enough for models to keep improving on that. The next big capability jump is probably going to come from the models teaching themselves.”

“And so this is kind of our view and our, if you extend this a couple of years out, and this is where some people will probably have objection with us. We think that this will go so far. That we will kind of recycle all of the world's real data into higher qual…”

“And so that means you kind of can't do like a prediction. There's no Moore's law here where you can say, well, it'll get to that power of compute level at this, but at this point set aside the fact that we actually don't have enough data to give it 10 or a hun…”

“We have kind of an existence proof now that it's actually not that hard to do this and so you don't need to invest all that much in, in data, and you can use synthetic data and get a pretty good model out of that”

“Synthetic data is incredibly effective. It's now the majority of the data that we train on for creating something like command A.”

“It is. I think it's becoming more of Thing, I would say. It still has its challenges, mostly to generate diversity of data, but definitely something we're exploring.”