Everything Douwe Kiela said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Douwe Kiela: Incumbent AI firms promote existential risk to entrench regulatory advantage
“So the people who are pushing this narrative are really the people who are benefiting from this being the narrative. So these are the incumbent AI companies who are, who want to have either the market regulated. Right. Because then they benefit because they ca…”
Kiela: Core language model development is almost solved and plateauing
“It's not even really about language models anymore. That has almost been solved, right? That's kind of why you see things plateauing off a little bit as well.”
Kiela: Long-context LLMs are inherently incredibly wasteful
“Long context models are inherently incredibly wasteful. You're paying for all this compute, and that's maybe why some of the companies that are trying to really sell long context model, long context window models, they will make more money from that, right?”
Kiela: Believing open-source AI can match OpenAI is incredibly naive
“I would like it to be true that with open source, we could just keep up with all of that. But I think that's just incredibly naive. Open AI has this very deep understanding of how people want to use language models. Basically nobody else has, and they have thi…”
Douwe Kiela: The leaked 'no AI moat' Google memo is completely wrong
“I think in terms of data modes and maybe you, you've seen this come by actually, there was this Google memo from an internal Google employee who had written that open AI and Google have no mode. I think for me as a AI researcher, when I read that memo, I was l…”
Douwe Kiela: Getting struck by lightning is more likely than AI extinction
“Probably the chance of me getting hit by lightning, like right now is much higher than that happening.”
Douwe Kiela: EU AI regulation will completely destroy innovation
“What Europe is going to try to do is overregulate everything and just completely destroy
innovation.”
Douwe Kiela: AI will displace the majority human workforce within 5-10 years
“If you look at how OpenAI and Anthropic and these places define AGI, it's as systems achieving capabilities that allow them to effectively do the work of humans for the majority of economically valuable human tasks. Then we're not that far away from it. And so…”
Douwe Kiela: OpenAI and Anthropic are the Lycos and AltaVista of AI
“So if all the stars align, then OpenAI and Anthropic and all of these places, they had this great first generation technology. So they're kind of like the Lycos and Alta Vista of search engines. And the technology we have is more like PageRank. And that would …”
Kiela: DeepSeek proved frontier AI models can rely on synthetic data
“We have kind of an existence proof now that it's actually not that hard to do this and so you don't need to invest all that much in, in data, and you can use synthetic data and get a pretty good model out of that”
Kiela: DeepSeek's total development cost was at least 100x its $6M training
“So I would guess that they spent at least a hundred X The amount of that, that single training run, right?”
Kiela: Fine-tuning cannot inject new knowledge into AI models
“One common misconception about fine tuning is a lot of people think that you can inject new knowledge into a model using fine tuning. And that is not true.”
Kiela: Advanced RAG systems break down when scaling to a million PDFs
“You can build a very awesome demo on a couple of PDFs and things will probably work. But then you have to scale it up to a million PDFs, and then everything breaks down. And the reason for that is that a lot of these kind of advanced RAG systems still actually…”
Kiela: AI systems will probably never reach 100 percent accuracy
“When are we getting to a hundred percent accuracy? And I had to give them the bad news that probably never.”
Kiela: Companies Will Stop Trying to Build In-House AI Within Years
“Very often they think they can do it in house.
And so I think that belief is probably going to go away in the next couple of years where people realize that this stuff is a little bit more difficult than they,
Initially thought.”
Kiela: Top-Down Enterprise AI Budgets Will Dry Up After Failed Pilots
“That money is temporary and it's going to dry up. And so it's, we're going to have a couple of cool pilots and demos, and then at some point it doesn't work and we will move on to the next hype train.”
Kiela: Enterprise AI Demands End-to-End Retrieval Models Over Parametric Monoliths
“And so we think that's not what we now call a language model. So it's not one big parametric monster. It's something a bit more elegant that has the retrieval kind of built in. It's a retrieval augmented language model and is strained end to end so that it can…”
Kiela: Enterprises do not need model fine-tuning when RAG is available
“You don't have to fine tune your model. It feels very intuitive. We have this great data set. We own it. It's our data. So we need to do something useful with it. So we need to fine tune our own language model. And so the companies who are offering that servic…”
Douwe Kiela: AI Model Parameter Sizes Will Stop Growing Rapidly
“I think Sam Altman had this interesting quote where he was saying that he thought models would stop growing in size. GPT-IV kind of hit this ceiling. I think that's probably right, but not really because size doesn't matter. It's just that data size matters ev…”
Douwe Kiela: GPT-4 Will Disrupt Data Annotators Like Mechanical Turk
“One of the use cases I've been seeing now for GPT-IV is actually that people are using it to generate data and then they're training on that data with cheaper models. So GPT-IV might end up disrupting, not knowledge workers necessarily, but it might just disru…”
Douwe Kiela: Current AI breakthroughs would not happen without Meta's PyTorch
“So PyTorch really without PyTorch, none of this stuff would be happening right now. And so it's really like fundamental for all of the AI breakthroughs.”
Douwe Kiela: Artificial Specialized Intelligence will be achieved much faster than AGI
“And I think you're totally right that that's a much easier problem to solve much quicker and then slowly grow with the capabilities of these models.”
Douwe Kiela: Data size matters more than model size for AI performance
“It's just that data size matters even more than model size. And I think the Lama paper out of Meta really brilliantly showed this. Where if you train a smaller model on more data for longer, then you get a better model. So you get more bang for your buck if yo…”
Douwe Kiela: GPT-4 will disrupt Mechanical Turk before knowledge workers
“And so, so GPT-IV might end up disrupting, not like knowledge workers necessarily, but it might just disrupt like mechanical Turk and is just a, an annotator on steroids.”