foundation model

“We coined the term Foundation Models because we thought it, there was something that was happening in the world that was, that somehow large language models didn't really capture the significance. And it was not just about language, it was about images and mul…”

“And now today I think it's the dynamic is, is quite different because it's no longer academia's job isn't just to get things to work because you can do that in other ways. There's a lot of resources going into big tech companies where there's if you have data …”

“Certainly these are powerful technologies and could have extreme social consequences, but there's a lot of more near-term issues.”

“And this was a paradigm shift, in my opinion, because it changed the way that we conceptualize Machine learning and NLP systems from these bespoke systems where you're, it's trained to do question answering, to train to do this, to just a general substrate whe…”

“And what we're seeing now is sort of a retreat of that open culture where models are now being only accessible via APIs. We don't really know all the secret sauce that's going behind them, and there's sort of limited access.”

“The analogy I like to think about is, you know, nutrition labels or any sort of specification sheets on electronic devices. There's some sort of, ah, obligation. I think that you know, producers of some products should have to make sure that their product is u…”

“It's going to be five X cheaper to train the same size model the second time around, like, once you've made your mistakes and know what you're doing.”

“So my take is that foundation models will all be open source for the deep learning phase.”

“I think that is also going to be something where players like Epic are going to be able to partner with existing models and I think potentially deliver real value there.”

“My new insight is there's going to be a lot of great AI native companies that are built on top of this new infrastructure. Let's call it like cloud two point O, which is this foundation models. These are the new backends that are going to power the most exciti…”

“It doesn't feel crazy to think that a good foundation model could outperform more traditional approaches by, I don't know, a hundred bips, 200 bips.”

“I think on the foundation model side, from a venture capital or angel investor perspective I think we're going to see fewer new language models But we should see models in a lot of other areas and, you know, we have new things happening in music. We talked a l…”

“Eventually the entire UI will go away once agents get good enough, right? And so I think this is kind of the intermediary step of human in the loop. And eventually as agents get smarter and smarter, and that's going to be through breakthroughs in the base mode…”

“Today the models is the front and center thing that everybody pays attention to, but I think increasingly it'll become a smaller percentage of the thing that people pay attention to”

“The API costs have dropped something like 200 X. In the last 18 to 24 months or something along the, along that order of magnitude, David on my team actually pulled together a chart of all the pricing for all the various models and what that looks like over ti…”

“I think especially at the early end of the market, I'm like, well, you know, really high quality opportunities are there's plenty of capital for them. I keep discovering that the capital markets are much deeper, and we should talk about this, are much deeper t…”

“I think one of the more, more interesting things is that you have convergence not just on capability, but also in the, like, product surface areas. Like, most people have search, they have a research product they have reasoning in the models.”

“I think the challenge right now is that if you're building a wrapper effectively as an AI, as a service company, and all your wrapper does is enhance the capabilities of a model or put some guardrails around it, then your biggest risk is the model slowly expan…”

“I think you know, a third area is the next set of foundation models are going to come, and by that I don't mean the Neolabs and the Next Gen LLM, which of course will happen, but I mean, physics, materials, science progress by models, math progress. And I thin…”

“And then from an AI standpoint, you know, I feel like we're generally okay with the investment. I mean, there's so much money going in there that that's like a well-funded area, both the foundation models and the derivatives that are trying to do specific apps…”

“To the extent that that is encoded in a model, I think a lot of their business will be at risk, but to the extent that it is encoded in workflows that is where they will be able to develop mode.”