The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 12 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Mann: Output verification will be the primary blocker for enterprise LLM deployments
“I think that's gonna be one of these evergreen problems, ah, for LLMs, and we're gonna, you know, keep trying to chew on that, ah, for a while, but that's gonna be the big blocker of a lot of the further deployments.”
Mann: GitHub Copilot saves developers about 40 percent time on new code
“They've done some studies, and it's, they, studies suggest that it saves about 40% for new code for developers to write new code.”
Mann: Every software application user interface will eventually integrate an LLM
“You can imagine that every point of interface with software application, there's gonna be a point to have a large language Model in that point of interface, and I think all of those will be interesting and useful”
Mann: Human intelligence is overrated, making the path to AGI more plausible
“I kind of think human intelligence is a little bit overrated. So I think I'm a little more bullish on AGI. Someone wrote, you know the large, I think it was Ilya Sutskever, you know, maybe the largest models are slightly conscious. I'm not sure I disagree. So …”
Mann: AI breakthroughs are an overnight success seven years in the making
“It, for me it feels like an overnight success, seven years in the making.”
Mann: Open source culture in academic machine learning accelerated AI innovation
“The academic machine learning community happened to be one that shared quite a bit. And so companies like Hugging Face and, you know, Google's Colab, you know, enabled sharing of code sharing of technology, and this just really increased the pace of innovation”
Mann: GPT-3 was vastly better than decades of prior NLP systems
“Two years ago, I remember being shocked at GPT-III when it was released, because it was just so, I'm an old NLP guy, and it was so vastly better than anything that, that we could build for decades.”
Mann: LLMs will outperform decades of prior enterprise search techniques
“When I look at the, you know, all of the LLM technology, large language model technology, it seems like it will get closer than any of the things that, that, you know, the field has been doing for the past few decades.”
Mann: GitHub Copilot is likely the most widely deployed LLM application
“So far the most widely deployed application of the large language models is probably Copilot.”
Mann: Unoptimized model pipelines mean AI development is still in early stages
“I think we're still early. I think On, on, on, for two reasons. I think one is, ah, the, you know, to Melanie's point, the amount of tuning and optimization across the entire pipeline is, is really premature.”