The Ledger, every show

“I think that, like, this is going to be the dominant paradigm. I actually think that, like, probably purpose-built re-rankers will go away, and the same way that, like, purpose-built, they'll still exist, right? Like, if you're at extreme scale, extreme cost, …”

“I think, like, there's a few things that I think might be true about retrieval systems in the future. So, like, number one, they just stay in latent space, they don't go back to natural language.”

“Diminishing, they're clearly diminishing marginal returns, right? We're sort of spending 10 X on compute. We're not getting 10 X or better models, at least evidently not yet.”

“It's a, you know, technology is probably as important as the invention of electricity. It will probably, you know, bring about a increase in GDP that is on the order of the industrial revolution or greater.”

“Like it is very possible the poorest people on earth today, or, you know, in 10 years, we'll have access to better healthcare better legal representation you know, better financial services than, like, billionaires have today.”

“We've like sort of stress tested like frontier models and their ability to search. And they are not actually that good at searching.”

“Like, the idea that there's going to be, like, a lot of offline compute and inference under the hood that helps make AI systems continuously self-improve is a sure bet.”

“And we think that both certainly transactional has to be the case because it is a online database. It's gonna sit in the loop of applications. Again, you've already seen demos of this happening tonight. But also to make this technology useful for developers, y…”

“And so, like, I think pruning is also going to be, like, really, it's already becoming a thing, right? But, like, letting models, like, self-prune their context windows.”

“For many years running, Chrome has been the number one used project broadly, but also within communities like LinkChain and Llama Index.”

“And yeah, they're using context caching and that certainly helps, but like their cost and speed, but like isn't helping the context raw problem at all.”

“I think like today, 90 plus percent of it in enterprises is retrieval event generation, or it's, you know, using retrieval, it's sort of a chat on top of unstructured data.”

“So I think that use case, even just the Chat Your Data use case, truly will go to the ends of the earth.”

“So, there's lots of stuff around steering language models at the embedding layer itself, and not using text but this is not yet exposed To at least closed source models, so.”

“We think the database will be, like, much thicker than it's been before.”

“There'll be language models running inside the application code, obviously, language models running inside the database as well or large models more broadly, multimodal models will run, you know, everywhere as well.”

“I certainly think probably most enterprises, organizations, companies on earth will have brought language models Into the company, probably in pretty meaningful ways. At minimum, the customer service department, the sales department, ops, back end, legal and h…”

“And obviously I'm incredibly proud that it exists today and that it's like serving hundreds of thousands of developers and they love it, but it was hard to get there.”