The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 16 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 0 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Schmidt: Decentralized training will force NVIDIA to redesign chips around VRAM ratios
“What might happen sooner would be a redesign of the types of chips that NVIDIA or someone would make. Okay, under this model, we can dedicate more VRAM versus, there's like this question of how much VRAM versus how much processing power is on a die, and that, …”
Schmidt: Compute chips in NVIDIA's RTX 4090 and H100 are almost identical
“I think people don't actually realize that like a forty-ninety and like an H 100 are in a lot of ways the same card. For the non-gamers in the room, explain the forty-ninety. The chip that's inside of them is almost identical. The chip, the actual compute chip…”
Schmidt: Consumer gaming GPUs will become the sweet spot for distributed training
“Because you're able to distribute it so wide, I think the gaming GPU angle is really going to be like the sweet spot. If, as long as there's continued to be sort of like higher end gaming GPUs, and those are on comparison with the high end training GPUs, even …”
Schmidt: Decentralized training of 400B parameter AI models is solvable by 2025
“I think it still is, it would still be, you know, like a next year sort of environment thing that we would have to do. There are some scaling problems, or not scaling problems, but technical things about how you shard the model, because at that point you get t…”
Schmidt: DisTrO will eliminate data centers' InfiniBand dependency before edge AI dominates
“I think there's, you know, immediately coming out. What you'll see is the ability for even centralized actors who might have multiple data centers to now, like just use them in a more efficient way. Like just have N equals two, you know, like anything. And eac…”
Schmidt: ChatGPT, Llama, and DeepSeek use Nous Research's YaRN context extension
“Bone here is the lead author of a method we developed called YARN, which is a context window extension method that we released and did the research on. It is now used by every, every model you use nowadays, everything, everything Chachipiti, Lama, DeepSeq, all…”
Jeff Schmidt: Hermes pioneered synthetic data training before it was standard
“So Hermes was very early to the idea that you could have synthetic data, which is that you could actually make, you could make a better model by taking an AI model, having it generate words and text, and then training a new a model on that output. This is now …”
Schmidt: Fewer than ten organizations worldwide can train Llama-scale AI models
“Yeah, I mean, I would, it would probably be in the number of ones on my hand and it probably wouldn't use all my fingers, you know. Yeah, I mean, you basically have, OpenA, Anthropic, Meta, X, Google, and then you have a few Mistral, and then Deep Seek and a c…”
DisTrO's performance advantage over AdamW widens as models scale up
“What we have seen empirically is that as we make it bigger, the differential between distro and MW actually gets wider.”
Jeff Schmidt: Open-source AI lags closed AI providers by 1 to 1.5 years
“It seems that we're in the open source space. We're always like a year playing catch up, like a year, it's like a year and a half, a year and a half behind like the closed providers.”
Schmidt: Current AI training requires all GPUs in the same datacenter
“When it comes to training models, the current paradigm for training models requires that all of the GPUs that train the model, these, you know, these computers that do the training, they all have to be like in the same room.”
Nous Research replicates DisTrO training results using Allen AI's OLMo framework
“And we've re-implemented now a third time in their framework, and we're able to reproduce their training run exactly, and then did it again with Distro, got the exact same results we got with Natron and stuff.”
Schmidt: Zeroth-Order Optimization Requires 1,000x More Computation Than Backpropagation
“What we discovered is that backprop is still being like, you really do still need to be doing back propagation to find the optimal point of the loss. And it's just like zeroth order is like, what, like a thousand, like it worked, but it was like, you needed li…”
Schmidt: Elon Musk's xAI has acquired 100,000 NVIDIA H100 GPUs
“I think Elon's got a hundred, a 100,000 H 100 now.”