model

“And I think that what that's going to unlock is you're going to be able to scale up these models and you're going to see qualitatively different behaviors from what you've seen so far.”

“The way that I think of these models, a super smart, you know, fresh grad right out of college that can do anything pretty well, but it has no idea what to do, you know, and the missing step is giving me the ability to teach it how to do work that I don't want…”

“I think maybe the thing to watch out for is things where, like, they work now with a huge amount of work, like, to build up a scaffold, but the next generation, you're not gonna need the whole scaffold you built up.”

“No, they didn't do anything, you just adapted to the new standard, and now your expectations went up, but the model is still the average.”

“I think a lot of the work to your point is the, how do you organize the context for the model? And you can use a model to help, but that is the bottleneck for most things.”

“I think evals, they outlive the harness a little bit, but not quite that much. Like, an eval might live for maybe one, two, three model generations, but nowadays the, you know, we're on the exponential. The model is improving so quickly, very often we just sat…”