The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 32 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Hotz: GPT-4 is an 8-way mixture model with 220B parameters per head
“GPT-IV is two hundred twenty billion in each head, and then it's an eight-way mixture model.”
Hotz: Meta attracts researchers who want to publish while OpenAI keeps ideologues
“OpenAI can keep ideologues who, you know, believe ideological stuff, and Facebook can keep every researcher who's like, dude, I just want to build AI and publish it.”
Hotz: Analog computing for AI won't work and clockless chips aren't practical
“Analog computing just won't work. And clockless computing sure, it might work in theory, but your ETA tools are, maybe AIs will be able to design clockless chips, but not humans.”
Hotz: Systolic arrays are the wrong architectural choice for AI chips
“I think systolic arrays are the wrong choice. Systolic array, I think they have systolic arrays because that was the guy's PhD. And of course Amazon makes... They are very power efficient, but it becomes hard to schedule a lot of stuff. On them, if you're not …”
Hotz: Tinybox will be 5x faster than H100 systems per dollar
“For 90% of most companies model training use cases, the tiny box will be five X faster for the same price.”
Hotz: Machines will replace all human labor in about 20 years
“I'm a believer that machines are gonna replace everything in about 20 years.”
Hotz: Maximally compressed human brain represents only a couple gigabytes
“Quantization is a poor man's compression. I think we're only talking really here about, like, maybe a couple gigabytes, right? And then if you have, like, a couple gigabytes of true information of yourself up there, cool man. Like, what does it mean for me to …”
Hotz: Real AI alignment problem is corporate and government misalignment
“I think it's actually not a question of whether the computer is aligned with the company who owns the computer. It's a question of whether that company's aligned with you or that government's aligned with you. And the answer is no. And that's how you end up de…”
Hotz: Tinygrad runs all ML models with only 25 primitive operations
“Tiny grad is, we are going to make a risk offset for all ML models. And yeah, it can run all ML models with basically 25 instead of the two 50 of XLA or PrimTorch. So about 10 X less complex.”
Hotz: Google TPUs are the only successful non-Nvidia training chips
“The only company, there's one other company aside from Nvidia who's succeeded at all at making training chips... Mid journey is trained on TPU, right? Like a lot of startups do actually train on TPUs, and they're the only other successful training chip aside f…”
Hotz: AMD CEO Lisa Su sent pre-release ROCm 5.6 to fix panics
“Lisa Sue reached out, connected with a whole bunch of different people. They sent me a pre-release version of Rock M 5.6. They told me you can't release it, which I'm like, okay, Why do you care? But they say they're going to release it by the end of the month…”
Hotz: Best chatbots will be smaller models with 1,000 training runs
“I don't think that the best chatbot models are going to be the big ones. I think the best chatbot models are going to be the ones where you had a thousand training runs instead of one. And I don't think that the interconnect bandwidth is going to matter that m…”
Hotz: Third company will build an AI girlfriend product
“The third company's the first one that's gonna build a real product, and that product is A girlfriend? No, like I'm dead serious, right? Like this is the dream product, right? This is the absolute dream product.”
Hotz: Merging with machines requires an AI companion, not Neuralink electrodes
“So I don't need to put, you know, electrodes in my brain to merge with a machine. I need an AI girlfriend, right?”
Hotz: tinycorp will eventually partner or manufacture its own chips
“So I'd like to start another organization that eventually in the limit either works with people to make chips or makes chips itself and makes them available to anybody.”
Hotz: Tinygrad runs OpenPilot in production 2x faster than Qualcomm's library
“TinyGrad is used to run the model in OpenPilot. Like right now, it's been live in production now for six months. And TinyGrad is about two X faster on the GPU than Qualcomm's library.”
Hotz: Tinygrad could replicate PyTorch's API in two engineer-months
“Replicating the PyTorch API. Is something I can do with a couple, you know, like an engineer month or two.”
Hotz: Nobody has successfully trained models in INT8
“No one's gotten training to work with Indate yet. There's a few papers that vaguely show it, but if you're training, you're going to need BF-sixteen or float-sixteen.”
Hotz: None of 20 tested PCIe extenders work at PCIe 4.0
“No PCI extender I've tested and I've bought 20 of them works at PCIe four point out. So you're going to need PCIe redrivers now.”
Hotz: Except for Apple, secretive tech companies hide underwhelming tech
“Whenever a company is secretive, with the exception of Apple, Apple's the only exception, whenever a company is secretive, it's because they're hiding something that's not that cool.”
Hotz: Training on internet cross-entropy loss yields mediocre AI responses
“The problem is, if your loss function is categorical across entropy on the internet, your responses will always be mid.”
Hotz: RLHF models adopt customer support personalities
“I don't like the RLHF models. I don't like the tuned versions of them. I think that they become, you take on the personality of a customer support agent, right?”
Hotz: Google's TPU compiler is a closed-source 32MB binary blob
“Not only is the chip closed source, But all of XLA is open source, but the XLA to TPU compiler is a 32 megabyte binary blob called lib TPU on Google's cloud instances. It's all closed source.”
Hotz: Tinygrad beats CoreML on ONNX tests and will soon pass ONNX Runtime
“We're below Onyx runtime, but we're beyond CoreML. So, like, that's, like, where we are in Onyx support now, but we will pass Onyx runtime soon, because it becomes very easy to add ops, because of how, like, you don't need to do anything at the lower levels.”
Hotz: Tinygrad is about 5x slower than PyTorch on Nvidia GPUs
“The correctness for both forwards and backwards passes is there, but on Nvidia, it's about five X slower than PyTorch right now.”
Hotz: Intel GPUs feature stable kernel drivers and public register docs
“Intel GPUs have a stable kernel driver and they have all their hardware documented. You can go and you can find all the register docs on Intel GPUs.”
Hotz: Consumer AMD GPUs lack peer-to-peer support
“If you have a consumer AMD GPU, they don't support peer to peer.”
Hotz: Halving GPU power yields 80% of peak performance
“Now, you can limit power on GPUs and still get, you can use like half the power and get 80% of the performance. This is a known fact about GPUs”
Hotz: Qualcomm SNPE cannot run transformers due to missing outer product ops
“Qualcomm's S and PE can't run transformers for this reason. So most matrix multiplies in neural networks are weights times values, right? Whereas you know, when you get to the outer product in in transformers, well, it's waste times weight. It's a, it's values…”