The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 11 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Jack Morris: Fundamental AI science shifted to companies due to academic compute limits
“That's when I think things really started to change in terms of the types of questions you wanted to ask can't always be answered with academic resources. So a lot of the like fundamental kind of like boundary pushing and AI science moved into companies.”
Jack Morris: Most AI research was previously open, but is now closed
“Most stuff was open. Now most stuff is not open.”
Morris: New embedding inversion model exactly recovers 90% of source text
“Like we ended up building a system that can do this quite well, like taking an embedding and I think our highlight number is like at a certain length, like a long sentence length, we can get 90% of the text back exactly.”
Morris: Language models hit a hard memorization plateau regardless of dataset scaling
“Like, no matter how you scale the training size, you hit this like perfect, perfect ish plateau in auto memorization, which we call the model capacity.”
Morris: 32-bit transformer models store only 3.6 to 3.9 bits per parameter
“Transformers that are trained in 32 bit precision, we approximate can store about 3.6 bits of information to maybe 3.9 bits somewhere in there per parameter.”
Morris: The next AI paradigm shift will stem from an unused data source
“And so whatever the fifth thing is, whether it's Video or embodied AI or some kind of crazy innovation on reasoning models. Whatever comes next will probably be some type of new data source that we're not using yet.”
Morris: Top AI graduate programs do not teach multi-node distributed training
“Oh, to be clear, they don't teach you anything, like anything, like if you see a paper coming out from even, you know, Stanford, they're probably the best school in AI if you had to choose. And it's not like they're learning how to do like multi-node distribut…”
Morris: vLLM and SGLang are here to stay and will grow more complex
“I also think, ah, VLLM and SGLang seem, like, really good and important and here to stay. Like, they'll probably just get larger and more complex to accommodate future systems”
Morris: Embedding inversion requires access to and repeated queries of the encoder
“Like none of the vector to text stuff works unless you have this assumption of like knowing the encoder and also being able to make a lot of queries to it.”
Morris: CycleGAN mapping aligns disparate model embeddings without paired data
“We took it and we applied it to model embeddings where instead of zebras and horses, we have like BERT embeddings and GPT embeddings, or like two completely different models with different architectures. So I think these are GTR, which is a T five based retrie…”
Morris: LLaMA architecture will likely store more information per parameter than GPT
“Maybe even if we tested this with LALAMA architecture, like, there's sort of like a GPT++ architecture, like, I would guess that can store better data just because the kind of numerical flow is a little bit better, the nonlinearities are maybe, like, A little …”