The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 18 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Kolter: AI models contain an unpatchable buffer overflow vulnerability
“This is sort of like these models have a buffer overflow in all of them that we know about, and most importantly, that we don't know how to patch and fix. We don't know how to fix this yet with models”
Kolter: AI progress faces compute bottlenecks rather than a data shortage crisis
“We are nowhere close to hitting the limits of available data in these models. Arguably, we're unable to process it because we don't have enough compute and things like this, but we're nowhere close to data limits in other senses.”
Kolter: Multimodal AI data is abundant but constrained by compute bottlenecks
“There are massive amounts of data available, And I think we have not yet figured out how to properly leverage those due to either limitations of compute. I mean, you have to process all that data and it does take, we don't have current models to do this very w…”
Kolter: Larger AI models perform better on fixed datasets without plateauing
“But it is also true that if you just take a fixed data set and run over it multiple times, if you use a bigger model, it will often work better, right? So I think that we have not really reached the plateau there.”
Kolter: Training custom AI models from scratch will become economically unviable
“Companies that are right now thinking about training their own models and things like this, and it's just sort of the default that of course you would do this, that this must, won't be an economically viable thing to do. In the future, and so it won't happen a…”
Kolter predicts AGI will arrive within 4 to 50 years
“And I am massively uncertain as to when this will happen, but a massive shift that I've undergone is I think this will probably happen in my lifetime. I think the answer to AGI has always been in academia, not in my lifetime. And the timeframe I give this righ…”
Kolter: Winning companies will leverage workforce with AI, not fire them
“The winners in this new world Will the companies that say survive and thrive and become dominant in this new world? Not talking about the AI companies for now, talking about sort of the rest of the companies that the ones that people worry that they're going t…”
Kolter: Video will soon cease to be trusted as objective evidence
“Video is a short blip where we sort of think that there's some objective evidence for a hundred years of our history, and that's going to be now, that's no longer true pretty soon.”
Kolter: Existing AI models can already analyze code to find security vulnerabilities
“I think this is essentially already solved in many cases by these models that can already solve and analyze code to find vulnerabilities.”
Kolter: AI models will eventually reach capabilities that make open-sourcing unsafe
“I think there will come a time when a certain capability, a certain ability of these models Reaches the point that should give us pause when it comes to just turning these things over to whoever and what, however, they want to use them.”
Kolter: We do not understand how AI models work internally
“We do not understand how these things work internally, the possible correlated failures, the possible attack factors, all sorts of things. We don't understand it.”
Kolter: AI will accelerate trust erosion until people believe nothing they see
“The real negative outcome is that people are not going to believe anything that they see. It didn't even need AI to get there, but AI is absolutely an accelerant for this process.”
Kolter: Current AI models cannot reliably follow developer specifications
“Right now, the mod, the AI models that we have, for lack of a better phrasing, are not able to reliably follow specifications.”
Kolter: Public AI models are trained on around 30 terabytes of text data
“Public models are trained on the order of, you know, 30 terabytes of data or something like this, right? So 30 terabytes of text data.”
Kolter: AI foundation model landscape will likely undergo market consolidation
“And I think there will be most likely consolidation, but I'm not quite sure how it will play out.”
Kolter: Current AI products fail to let workforces maximize AI potential
“Do we have AI products that are able to be maximally used by workforces? And the answer to this is right now is no. Clearly there is a gap between what People could use these things for and what they're using them for right now.”
Kolter: RAG systems will remain essential despite fine-tuning advances
“RAG based systems are so Are so common here, and so, and probably will remain, even with the advent of fine-tuning availability, they're going to remain a useful paradigm.”
Kolter: Enterprise cloud LLM adoption will mimic cloud storage migration
“This is not in certain use cases, any riskier than just having your data in the cloud to begin with, which all of them typically do. They've all moved that way. So I think this will just happen naturally with progression of time.”