The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 12 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Liu: Cody matches GitHub Copilot completion acceptance rates using open-source StarCoder
“Like today, Cody uses StarCoder for inline completions, and with the benefit of the context that we provide, we actually show, like, comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completi…”
Liu: Pure transformer models are insufficient to support autonomous AI agents
“We're actually a little bit, I think, more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models.”
Liu: Reliable AI coding workflows require search-based algorithmic backbones
“The way that we get to this, like, more reliable, multi-step workflows that can do things beyond, you know, generate unit test is, is, it's really gonna be, like, a search-based approach, where, where you use an LLM as, kind of, like, an advisor or a proposal …”
Liu: Open-source AI models are currently state-of-the-art for code completion
“Yeah, I mean, for completions, open source is, is state of the art right now.”
Liu: Reliable single-step generation is a strict prerequisite for true AI agents
“If you want to get to the point where you can actually be truly agentic or like multi-step automated a necessary part of that is like the single step has to be robust and reliable.”
Liu: AI coding assistants must pull context beyond Git repositories to succeed
“And I don't think the AI developer will be any different. It will need to pull context from all these different sources.”
Liu: Post-hype reality check will expose the limits of AI search techniques
“And I don't know, I think in the next year or two, maybe as like the, as we get past like the peak AI hype, we'll start to see the gap emerge or become more obvious to more people about like how, how, how many of like the newfangled techniques actually work in…”
Liu: Codebase complexity is software's bottleneck, not code generation speed
“The real problem of modern software development, I think is, is not how quickly can you write more lines of code. It's really about managing the emergent complexity of code bases as they evolve and grow, and how to get, how to make like efficient development t…”
Liu: No AI coding tool uses one model for completion and chat
“Everyone uses a range of model. No, like, no one uses the same model for, like, inline completion versus, like, chat because the latency requirements were.”
Liu: Long-context recall depends directly on needle-in-haystack training loss
“The skill with which models are able to take advantage of context is always going to be dependent on how that factors into the impact on the training loss, right? So like, If you want long context window models to work well, then you have to have a ton of data…”
Liu: Deterministic graph context eliminates common AI code completion type errors
“Turns out that works really well. Like, you can eliminate a lot of type errors that, that AI coding tools make just by pulling in that context.”
Liu: Synthetic data and task-specific fine-tuning provide alpha for code automation
“I feel like most models today, they still use, like, combination of, like, the stack and the pile as, like their training corpus but you can only stretch that so far. At some point, we need more data and I don't know. I think there's still more alpha in, like,…”