The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 6 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Claude 3.7 performed worse than Claude 3.5 on Convex evals
“For example, we just tried clod three seven and it performs worse than clod three five on convex evals with the same prompting.”
Cursor Composer solves Convex benchmarks but fails on alternative backends
“We did notice that I mean, with convex, it pretty much autonomously just solves the first two tasks. It has a few round trips on like some errors that are only show up and playing with the front end. And then it's able to complete this files task and kind of g…”
AI hallucinates Convex code due to API similarities with Firebase
“I mean, I think one thing that's kind of interesting given these, you know, models inherent internal structure is that we noticed that a lot of hallucinations come from parts of our API that are like very close to Firebase, but not exactly Firebase.”
Claude 3.5 in Cursor autonomously codes for 10 plus minutes
“When it has the right feedback in cursor composer, and this was even on cloud three, five, it can just autonomously code for a 10 plus minutes and it can fix its own bugs. It can get to the point where it's like pretty much a fully working app with just an ini…”
End-to-end type safety and tight feedback loops improve AI agent performance
“It's like, you know, having just really tight feedback loops and having strong guardrails, like in convex, that's like end to end type safety, but that's like, you know, could take it in many, many different forms, right? Like making it so code is very cheap a…”
Jayakar: AI Models Stalled on Convex's Subtle Distinction Between Null and Undefined
“Convex has like a pretty subtle distinction between null and undefined, like just similar to JavaScript. And we noticed that like, because this is a subtle thing that's unexpected, it was The model got stuck on it and couldn't even figure it out. And so this i…”