foundation model

“It shows that you can take these massive kind of generalist foundation models, And not only can you tune them using that kind of programmatic labeling to be more accurate on a given task that you need to be really accurate on, but you can then distill down ver…”

“Most of these foundation models, like a GPT-IV, five, six, seven, are going to need to be customized on your specific data and knowledge and workloads.”

“In most real-world use cases and we'll get into how I define that in a bit, there's still a last mile to be traversed. So it's not that they're overhyped, per se, they're just not the complete solution.”

“I think it's very difficult to create anything relevant if we were to train our own foundation model.”

“We got priced out.”

“I think we will get to a point where it fully replaces.”

“But we have a phrase at poolside, which is over time, everything collapses into the models.”

“Foundation models today can't do it, and we don't yet have a path In them successfully doing so in a way that, like, really improves it.”

“And the problem is that as far as we can see, there is no winner takes all effect or network effect in a foundation model.”