foundation model

“Like to me, the barriers to entry of training, one of these models are mostly capital and talent. Like the people needed are still very specialized and very smart and you need lots of money to pay for GPUs. But beyond that, I don't see that much secret sauce, …”

“If it's like, Hey, like throw your data in here and we'll do like automations on top of it, like for everything, that's probably Hard to compete with whatever one of the foundation models might offer, but if it's like, hey, we're our, give us, like, your sales…”

“So it turns out that switching costs for foundation models is relatively low, and we often architect our software. Oh, AI Suite is open sourcing that my friends and I worked on. So it makes switching easier. Switching costs for the orchestration platforms is a…”

“For example, if you want to build a coding assistant, you don't nowadays develop something specifically for coding, but you develop and you build on models that were trained on large amounts of data, not just on code.”

“It might be that like the next stuff that like is soda on benchmarks is not the most expensive newest foundation model with the most like GPU training. It's like a swarm of lower cost cheaper models working together just like humans do to solve a problem.”

“This only works for simple cases today, and the reason why that's the case is because I think it's pretty fundamental limitation of the model that we have today, which is that they are not at the core model that take action in the world and see the consequence…”

“That all the leading foundation models are pretty poor at games, which is quite interesting.”

“I think that those kinds of interdisciplinary teams, especially if it involves the world of atoms as well there's not going to be a shortcut to that, at least in the foreseeable future. Those are areas that are pretty safe from just getting swarmed by whatever…”

“And I think a lot of those ideas, both from AlphaGo and AlphaZero are really, really relevant to where we are with today's foundation models. And I think a lot of that is what we're going to see of the advances the next few years.”