The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 7 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
New pre-training techniques will drastically boost base AI model capabilities
“The way that we used to do pre-training, maybe, like, you know, like two, a year ago or two years ago maybe, like, you know, diminishing return is, like, obvious, but I can see how new ideas are bringing, like, you know, fresh, fresh energy into the pre-traini…”
Fully automated AI self-improvement will eliminate human bottlenecks and trigger breakthroughs
“The moment that we had this full automation, I would say we can close the loop of self-improvement and then it becomes the Like, you know, the problems become like, you know, mostly providing compute for these models to actually do what they want to do. And as…”
AI model progress will alternate between pre-training and post-training breakthroughs
“We're going to be having a bit of a swing back and forth between pre-training and post-training.”
RAG will shift from universal use to handling long-tail distribution cases
“Maybe it changes in a way that, you know, like it doesn't need to trigger RAG for like everything, but I'm pretty sure that they're going to be some tail of the distribution that we're going to do RAG still for it.”
Real-world physical grounding will become the bottleneck for AI self-improvement
“As I said, you know, like soon, like the concept of data, like, you know, how to kind of like, you know, enable these models to kind of like, you know, be very good at like self-improvement becomes, how can I ground these models in, in real world? So this is d…”
Becoming a deep expert in a narrow subject will soon lose value
“Becoming absolute expert about a very specific subject, most likely Is not going to be useful in, in, in like the near future.”
AI data work will shift toward physical world grounding and interactive environments
“At the end of the day, I think, like, the work that we're doing on the data side most likely is going to shift toward building environments or making sure that these models can interact with physical words, and then it becomes more of a problem of, okay, how c…”