The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 8 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Malte Ubl: When Vibes and Eval Data Disagree, Vibes Are Right
“I think that the common quip that if the vibes and the data disagree, the vibes are probably right. It's true, right? So you have to like, be honest with yourself, like, do they agree and kind of iterate On them over time.”
Thick abstractions in nascent AI frameworks are usually wrong
“And so if you put a very thick abstraction, then it's probably going to be the wrong abstraction. You have to, so you have to be humble and say, okay, I need to stay low level. So that this can be flexibly used as trends emerge, right?”
Framework creators build ivory towers unless they build real applications
“The reality is that framework builders are usually not application builders. And so they built ivory towers that when they're hyper geniuses, or they get lucky, they're happy to be good. But like, but if you want to do this in a reproducible fashion with a hig…”
Ubl: Evals Function to Tell Developers Overnight Whether a Change Is Good
“The way I think about evals is essentially like, it's the thing that, that can tell me tomorrow whether my change is good. And I can operate without that knowledge, but it's super, super helpful.”
Ubl: Transaction systems require explicit or ad hoc workflow abstractions
“What's extremely common and essentially every transaction processing system that has ever created is that there either is an explicit abstraction for workflows in it, or someone made one ad hoc, because otherwise the thing just doesn't work.”
AI agents only succeed when scoped to narrow problem domains
“At this moment, they're, like, agents are both extraordinarily effective and still very ineffective. And you have to find the right problems. And then when you find the right problems, they are super magical. And if you wander beyond, then they don't work.”
Asking employees what they hate identifies high-impact AI agent tasks
“Basically where the idea is that you go around your company and you ask people like, what do you hate most about your job? And I really think it finds the sweet spot because it finds problems that are, they're boring because they're tedious and repetitive, but…”
AI product pivots must naturally extend a company's core identity
“And I think there's a lesson there to be learned for companies that haven't done the transition yet. Is that you have to do something that feels native to your company. And so the two big bets that we made early on, one being vZero and one being AISDK, I think…”