The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 11 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Bender: ChatGPT only mimics language form and fails genuine information needs
“What ChatGPT can do is it can mimic human language use across many different domains. And so it can produce the form of a poem. It can produce the form of a travel itinerary. It can produce the form of a Wikipedia page on the history of some event. It is an ex…”
Bender says large language models are parlor tricks exploiting human perception
“Right down at the bottom, you've got the fact that especially large language models are a technology that is, that's a parlor trick. It plays on our ability to make sense of language and makes it very easy to believe there's a thinking entity inside of there.”
Bender says AI search queries use 30 to 60 times more compute
“Each of those tokens has to be calculated individually, and so it's coming out one word at a time, and that is far more expensive. I think her number is somewhere between 30 and 60 times more expensive, just in terms of the compute, which then scales up for el…”
Bender says Microsoft abandoned net-zero carbon goals because of generative AI
“I would say that we've got Brad Smith at Microsoft giving up on the plans to become a net zero carbon since the beginning of Microsoft, and he said this ridiculous thing about we had a moonshot to get there, and turns out with generative AI, the moon is five t…”
Bender says most AI benchmarks fail to measure actual capabilities
“Most of the benchmarks that are out there are not reasonable. They lack what's called construct validity, and construct validity is this two-part test of the thing that we are trying to measure is a real thing, and this measurement correlates with it interesti…”
Bender argues AI bioweapon risks are speculative fiction distracting from real harms
“Bad actors could use it to, you know, more quickly design, you know viruses and stuff like that. That's still speculative, right? So anytime we are taking the focus away, it's like, has that happened? Right? This is still people writing science fiction, fan fi…”
Bender: Building general AI is not sound engineering practice
“So the ability to be general and here I'm thinking of the work of Dr. Tamik Gibru is not an engineering practice. That's an unscoped system. So what Dr. Gibru says is the first step in engineering is your specifications. What is it that you're building? If wha…”
Bender: AI Bots Cannot Have Judgment or Accountability
“A bot is not the kind of thing that can have judgment, nor is it the kind of thing that can have accountability for exercising judgment.”
Bender argues clinical note-taking is essential medical thinking, not clerical work
“Writing the clinical note is actually part of the process of care. It is the doctor reflecting on what came out of that conversation with the patient and thinking it through, writing it down, plans for next treatment. That is not something that I want doctors …”
Bender: AI coding assistants accumulate undocumented technical debt
“As coding assessments, assistants, so that's sort of a machine translation problem between natural language and some programming language and there I really worry about technical debt where you have, you know, output code that was not written by a person that'…”