The Wisdom Wall
10 quotable lessons, heuristics and mental models. Every one is playable at the moment it was said. No fortune cookies allowed.
“NVIDIA's moat in, in inference is actually A lot smaller on software, um, but it's a lot bigger on, hey, they just have the best hardware.”
“You can create data out of thin air almost, right? Um, in certain domains, right? And so this is the whole, the debate around scaling laws is, um, how can we create data?”
“why is Mark Zuckerberg building a two gigawatt data center in Louisiana? Why is, why is Amazon building these multi gigawatt data centers? Why is Google, why is Microsoft building multiple gigawatt data centers? Plus buying billions and billions of dollars of fiber to connect them together because they think, Hey, I…”
“So, the, the whole paradigm of training, you know, pre-training is, is, is not slowing down. It's just, it's logarithmically more expensive each, for each generation, for each incremental improvement.”
“When I do this with O-one, right, because it's doing that thinking phase of 10,000... it spends a lot of memory on generating this KV cache and reading this KV cache constantly. Now the maximum batch size, i.e. concurrent users I can have, is a fraction of that. One-fourth to one-fifth the number of users can currently…”
“building a chip is one thing, but building many chips that connect together, cooling them appropriately, networking them together, making sure that it's reliable at that scale is, is a whole host of problems that semiconductor companies don't have the engineers for.”
“if I just replace, like, six servers with one, I've basically invented power out of thin air, right? I mean, like, you know, in effect, because these old servers, which are six plus years old, or even, you know, they can, they can just be deprecated and put, so with CapEx of new servers, I can replace these old…”
“In fact, there's more inference in training than there is updating the model weights, because you have to generate hundreds of possibilities And then, oh, you only train on a couple of them, right?”
“We can't teach it what good art is. Because we have no way to functionally prove what good art is. We can teach it to write really good software. We can teach it how to do mathematical proofs. We can teach it how to engineer systems, because there are, while there are trade-offs, and this is not like, it's not just a…”
“Yes, the queries are expensive, but they're nothing close to the human, right? And so each level of productivity gain I get, um, each level of capabilities jump is a whole new class of tasks that it can do And, and therefore I can charge for that. Right. So this is the whole like axes of yes, I spend a lot more to get…”