The Ledger, every show

“It feels very clear to me that this type of logic is going to be built into the model. Anytime there is control flow complexity or uncertainty complexity, I think the history of AI has been to push more and more into the model.”

“And I think O-one is going to do that to agentic frameworks as well. Hey, I think To me, it seems very unlikely that the, you know, you and me sort of like sipping an espresso and thinking about how, like, different personified roles of people should interact …”

“The real gap is that software engineers who have a particular way of thinking, a particular set of biases, a particular type of workflow that they run, are going to be the ones who are doing AI engineering, and that the tools that were built for ML are fantast…”

“For probably 80 or 90% of the use cases that we see with people doing this, like very, very simple, I create a prompt, it calls some tools. I can like very ergonomically write the tools, plug into popular services, et cetera, and then just call them kind of li…”

“I will say in my own experience with customers as of the recording date today, which is September or something, yeah, very few of our customers are currently fine-tuning models. And I think a very, very small fraction of them are running fine-tuned models in p…”

“We still see an overwhelming majority of customers using OpenAI, but almost everyone is using Anthropic for the, and Sonnet specifically for their side projects, whether it's You know, via cursor or prototypes or whatever.”

“It hasn't been a smooth journey for people to get the capacity on public clouds that they're able to get through, you know, OpenAI directly. I mean, I think a lot of this is changing, catching up, et cetera. But it hasn't been perfectly smooth. And I think the…”

“Almost everyone that we work with has gone into this model that, that I, that actually exactly what you said, which is sprinkle intelligence everywhere and make it easy to write dumb code”

“Almost if not all of our customers have moved off of fine-tuned models onto instruction-tuned models and are seeing really good performance.”

“So we see very limited practical adoption of open source models, but I think more interest than ever.”

“LLMs and, you know, specifically embeddings are going to be core to how people actually query data, not, you know, traditional algebraic relational indexes.”

“What will really be disrupted is the OLAP workload. So relational, you can't just slap you know, semantic search and stuff into the architecture of a traditional data warehouse. I think that is actually a much deeper set of things that will need to change than…”

“Vectors are very difficult because they're a data inefficient representation, so the vector format in something like Figma Is choose up like many, many, many, many, many more tokens than HTML and JSX. So it's a very difficult medium to just sort of throw into …”

“There's a lot of cases now, especially with large context models, where in context learning just beats fine tuning.”

“Pre-Claude III, it was close to a hundred percent OpenAI.”

“I don't have any insider information, so I don't know about the hardware companies, but I do know for some of this, excuse me, for some of the software companies, they have high margins and they're making money.”

“I would say about 50% of the use cases that we see are what I would call like single prompt manipulations.”

“I'd say like probably 25% of the remaining usage is what you could call like a simple agent. Which is probably, you know, a prompt plus some tools. At least one or perhaps the only tool is a rag type of tool, and it is kind of like an enhanced, you know, chatb…”

“My conclusion is that this is a very real problem for a very small number of companies, and that is why Datadog, Splunk, Honeycomb, et cetera, built their own database technology, which is, in some ways, it's sad because all of the technology is a remix of pie…”

“And I think the way that enterprises will collect data and leverage it into, you know, these AI processes does not look like doing ETL on a data warehouse that's, you know, running in, in Amazon or something like that. I think it's gonna totally change.”

“Relational databases are perfectly capable of adding HNSW indices to them.”

“The interesting thing is that most customers, or actually I'd say a stronger statement, like all customers are quite afraid and reticent to just hand over the data that they use to do evals on to labs.”

“There's one library called Tantivi, which is a Rust re-implementation of the Lucene search index, which is a very popular and well-regarded and used search technology. It's the technology behind Elasticsearch as well. And that was the only piece of code that w…”

“DuckDB has a struct type, which is dynamically constructed, but it has all the downsides of traditional structured data types, right? So it's just not Like, for example, if you create, if you infer a bunch of rows with the struct type, and then you present the…”

“At Impira, I took kind of the popular advice, which is that developers are a terrible market. So we sold to line of business, and there are a number of benefits to that. Like, we were able to sell six- or seven-figure deals much more easily than We could at Si…”

“I think it kind of represents the future of AI engineering, one where You can spend a lot of time writing English and sort of crafting the use case itself. You can reuse tools across different use cases. And then most importantly, the development process is ve…”

“Zapier was our first user, and then Coda and Airtable quickly followed, and there was just no chance they would be able to use the product unless the data stayed in their cloud.”

“Now I would say every customer and probably north of 75% of the users that are running evals in brain trust are using the TypeScript SDK. It's an overwhelming majority.”

“Unambiguously, people are doing rag. So that one is, you know, it's like simple and obvious. Probably around 50% of the use cases that we see in production involve rag of some sort.”

“Especially, you know, Claude III-V Sonnet has really taken off.”

“Well, I went through this myself watching the technology that we built to do document extraction at Impura become, you know, totally irrelevant.”

“There's some companies that we talked to and their AI vendors are, it's literally OpenAI, AWS, and Braintrust and pretty much everything else has consolidated away.”

“We've seen like a massive surge of product manager, product managers and designers getting interested in participating in the eval process among our customers.”

“Every span in brain trust land, which is like a row of something that you'd log in brain trust, the average size is 50 kilobytes. In traditional observability, it's 900 bytes.”

“So a lot of the cool Silicon Valley companies, their multimodal workloads are Mostly images and nowadays videos. A lot of the more traditional companies have a lot of PDFs.”

“I think probably more than half of the evals that people do in Braintrust are LLM based.”