The Ledger, every show

“The issue really is the fact that these observability companies isn't actually doing observability for the system, it's just doing the LLM thing. Like I still end up using like Datadog, right? Or like, you know, Sentry to do, like, latency. And so I just have …”

“So much of it is changing that if you give control of these systems away too early, you end up ultimately wanting them back. Like many companies I know that I reach out or ones were like, oh, we're going off of the frameworks because now that we know what the …”

“I think a lot of these app layer startups should not be hiring MLEs because they end up churning.”

“Yeah, my money is on the rankers because you can do those so easily, right? You could just say, well, given the embeddings of my search query and the embeddings of the description, I can just train XGBoost and just make sure that I have very high, like, MRR, w…”

“Instead of doing like a react type reasoning loop, I think my belief is that we should be using like workflows, right? If we do this, then we always have a request and a complete workflow. We can fine tune a model that has a better workflow. Whereas it's hard …”

“I think the bigger challenge is like, okay, a hundred million dollars, probably pretty, pretty easy. It's just time and effort. And they have both like the manpower and the money to sort of solve those problems. I think it's just like, again, if you go the VC …”

“We, because, I mean, at this point, we would have, like, you know, three million pieces of inventory, over, like, a billion interactions between users and clothes. Any kind of fine-taining would definitely outperform like, some off-the-shove model.”

“In terms of whether or not, like, JSON mode is better, I usually think it's almost worse unless you want to spend less money on, like, the prompt tokens that the function call represents. Primarily because with JSON mode you don't actually specify the schema.”

“If you're running into issues where you have, like, 20 or 50 or 60 function calls, I think you're much better having those specifications saved in a vector database, and then have them be retrieved. So if there are 30 tools, like, you should basically be, like…”

“Overall, I'm like super happy with the anthropic models compared to the OpenAM models. Like, Sonnet is very cost effective. Haiku is, in function calling, it's actually better.”

“Embeddings really is kind of like the lowest hanging fruit, and using something like Instructor can really help produce a data structure, and then you can just use your computer science to reason about this data structure.”

“But to back to the instructor thing, I just don't think it's a billion dollar company.”

“I think something like DSPy can work because there are like very short term metrics to measure success, right? It is like, did you find the PII or like, did you write the multi-hop question the correct way? But in these like workflows that I've been managing, …”

“I think a lot more data science is going to come in versus machine learning engineering, because a lot of it now is just quantifying, like, what does the business actually want as an outcome, right?”

“Narrative matters a lot to human beings. And I think the recommendation system, that's really hard to capture. Like, it's easy to sell, it's easy to use AI to sell, like, a 20 dollar shirt, but it's really hard for AI to sell, like, a 500 dollar shirt.”

“Function calling lets you define the schema separate from the data and the instructions. And what this meant was you can kind of have a lot more complex schemas and just map them in Pydantic, and then you can just keep those very separate.”

“And so I just said, let me write, like, the most simple SDK around the OpenAI SDK, sorry, simple wrapper on the SDK, just handle the response model a bit, and kind of think of myself more like requests than an actual framework that people can use.”

“People want to build their own frameworks, but people don't want to build, like, JSON parsing.”

“In terms of an extraction workflow, I definitely think it's probably more helpful to have everything be a single schema. Just because you can sort of specify relationships between these entities, right, that you can't do in parallel function calling, you can h…”

“One of the things we found out about these language models is that not only can you define nodes, it's really good at figuring out what are nodes and what are edges.”

“I think, you know, people like Prefect and Zapier have a pretty good shot at doing a good job.”

“I think the higher agency person is more focused on like process metrics versus outcome metrics. Right? Like, from pottery, like, one thing I learned was, if you want to be good at pottery, you shouldn't count, like, the number of cups or bowls you make. You s…”

“So, agency lets you sort of capture the volume of experiments, and, like, experience lets you figure out, like, oh, that other half, it's not worth doing.”

“Or those are two sort of systems that I wish you before or Opus was actually good enough to just write me an essay, but most of the essays are still pretty bad.”

“I mean, the biggest one really was the fact that, like, I think for just four years I was so bearish on language models. And just NLP in general, I was just like, ah, like, none of this really works. Like, why would I spend time focusing on this? I gotta go do…”

“Like what you should write down is like, here are the conditions. This is the inputs and the outputs we tried the experiment on. And then one thing that's really valuable is basically writing down under what conditions would I revisit these experiments?”

“We actually were using Transformers at Stitch Fix, like, before the GPT-III model, so we were just using Transformers for recommendation systems.”