The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 15 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Kiela: Core language model development is almost solved and plateauing
“It's not even really about language models anymore. That has almost been solved, right? That's kind of why you see things plateauing off a little bit as well.”
Kiela: Long-context LLMs are inherently incredibly wasteful
“Long context models are inherently incredibly wasteful. You're paying for all this compute, and that's maybe why some of the companies that are trying to really sell long context model, long context window models, they will make more money from that, right?”
Kiela: DeepSeek proved frontier AI models can rely on synthetic data
“We have kind of an existence proof now that it's actually not that hard to do this and so you don't need to invest all that much in, in data, and you can use synthetic data and get a pretty good model out of that”
Kiela: DeepSeek's total development cost was at least 100x its $6M training
“So I would guess that they spent at least a hundred X The amount of that, that single training run, right?”
Kiela: Fine-tuning cannot inject new knowledge into AI models
“One common misconception about fine tuning is a lot of people think that you can inject new knowledge into a model using fine tuning. And that is not true.”
Kiela: Advanced RAG systems break down when scaling to a million PDFs
“You can build a very awesome demo on a couple of PDFs and things will probably work. But then you have to scale it up to a million PDFs, and then everything breaks down. And the reason for that is that a lot of these kind of advanced RAG systems still actually…”
Kiela: AI systems will probably never reach 100 percent accuracy
“When are we getting to a hundred percent accuracy? And I had to give them the bad news that probably never.”
Kiela: GPT-4o is already effectively a reasoning model via chain of thought
“I mean, you could argue that GPT-IV-O is also already a reasoning model. It just hasn't been trained on reasoning specifically, but, ah, it can do chain of thought, right? So if it can do chain of thought, it's basically already a reasoning model. It just hasn…”
Kiela: AI is heading toward specialized language models over generalists
“Where we're headed is that we will have more specialized language models.”
Kiela: Attention mechanism, not Transformers, was the real AI breakthrough
“So I would say, and maybe I'm biased because one of my best friends is, is on the original attention paper, but that was the real breakthrough. It's just like figuring out that you have this attention mechanism that actually allows you to yeah, to do a much be…”
Kiela: FAISS was the first vector database
“In the initial paper, we used a vector database or a face. So the words vector database didn't exist at the time. But so face was the first vector database.”
Kiela: Retrieval is the only way AI agents can handle proprietary data
“Really focused on retrieval because that's really the only way you get these agents to work on your data and your problems.”
Kiela: FAIR was the first team to build a generative RAG model
“Why RAG became the way you name these things is because it's generative, right? So we were the first ones to have a generative model there.”
Kiela: Enterprise RAG fails if complex document data is not properly extracted
“If you want to have a enterprise grade rag system, you are only as good as the data that goes into that rag system. So if you can't extract the data in the right way, so if you have like a sort of table structure and it has like nested information, you can't g…”