The Ledger, every show

“LLMs are kind of like a new operating system, right? So, the LLM is a new kind of a computer. It's setting, it's kind of like the CPU equivalent. The context windows are kind of like the memory, and then the LLM is orchestrating memory and compute for problem …”

“Mac minis, for example, are a very good fit for some of the LLMs, because it's all, if you're doing batch one inference, this is all super memory bound, so this actually works.”

“LLMs don't natively do this, and this is not something that has really been solved in the R&D of LLMs, I think. And so context windows are really kind of like working memory, and you have to sort of program the working memory quite directly because they don't …”

“We're now kind of like cooperating with AIs, and usually they are doing the generation, and we as humans are doing the verification. It is in our interest to make this loop go as fast as possible, so we're getting a lot of work done.”

“The code of the Vibe coding part, the code was actually the easy part of Vibe coding Menugen, and most of it actually was when I tried to make it real so that you can actually have authentication and payments and the domain name and the virtual deployment. Thi…”

“You can train it on a single node of H-one-hundreds in about 24 hours, and that costs roughly 600 dollars.”

“At the time of that post, we were using, in LL and that's in 30% less memory, and we were 20% faster in training, just the truth.”

“I think B to C should be the right Start point, because I don't think we can have a robot, like, crush grandma, is how I put it, sort of. I think it's like too much legal liability. It's just like, I don't think that's the right approach... So I think the best…”

“And then I would say also one last dimension of it is you benefit a ton from like the transfer learning between the different tasks. And in AI, you really want a single neural nut that is multitasking, doing lots of things that's very getting all the intellige…”

“The transformer hasn't changed that much. You know, we've added the rope positional and the rope relative positional encodings. That's like the major change. Everything else doesn't really matter too much. It's like plus three percent on a small few things. Bu…”

“That's the thing with AI. I feel like a lot of them, a lot of these capabilities are just kind of like prompt away. So you always get like demos, but like, do you actually get a product? You know what I mean? So so in this sense, I would say the demo is near, …”

“I don't actually want to live in a world where lineage like matters too much, right? So I'm hoping that AI can help you destroy that structure a little bit. It feels like kind of gatekeeping by some finite scarce resource, which is like, oh, there's a finite n…”

“It takes effort, but it's effortful, but it's also kind of fun. And you also have a payoff of like, You feel good about yourself in various ways, right? And I think education is basically equivalent to that. So that's what I mean when I say education should no…”

“You would feel nervous when your GPUs are not running. Like you have GPU capability and you're not maximizing the available flops to you. But now it's not about flops, it's about tokens. So what is your token throughput and what token throughput do you command…”

“So it controls all of my lights, my HVAC, my shades the pool and the spa, and also my security system. So I have a camera pointed outside of the house, and anytime someone rolls in, I have a Quinn a Quinn model that looks at the videos.”

“I do think that if you had an autonomous loop over those metrics, there's gonna be a lot of like good harding going on where the system will like overfit to those metrics.”

“As we made the autopilot better, basically the neural network grew in capability and size, and in addition to that, all the C++ code was being deleted and kind of like was And a lot of the kind of capabilities and functionality that was originally written in o…”

“We're kind of like in this 19 sixties-ish era where LLM compute is still very expensive for this new kind of a computer, and that forces the LLMs to be centralized in the cloud, and we're all just sort of thin clients that interact with it over the network, an…”

“They display jagged intelligence, so they're going to be superhuman in some problem-solving domains, and then they're going to make mistakes that basically no human will make, like, you know, they will insist that 9.11 is greater than 9.9, or that there are tw…”

“I think this will end up being like a gateway drug to software development.”

“And I do think that the current models are wasting a ton of capacity remembering stuff that doesn't matter. Like they remember SHA hashes.”

“And one thing I'd really like to do in LN.c is I just want to keep things simple. I don't want to create a tensor abstraction. I don't want to create any abstraction, really. It's just float arrays and operations on float arrays.”

“PMPP is actually quite good but also, I think, still kind of like mostly on the beginner level, because a lot of the CUDA code that we ended up developing in the lifetime of the LMC project, you would not find those things in, in this book, actually.”

“A lot of lmc just kind of, like balancing the improvement and speed with the complexity of what you're actually introducing, and so I've actually rejected a lot of PRs because of that, because the code starts to get crazy, and I think that decreased the amount…”

“Actually, like, end-to-end driving, when you're just imitating humans and so on, you have very few bits of supervision to train a massive neural net. And it's too Too few bits of signal to train so many billions of parameters. And so these intermediate represe…”

“In terms of the transfer from cars to humanoids, it was not, not that much work at all. And in fact, like the early versions of Optimus the robot it thought it was a car, like, because it had the exact same computer. It had the exact same cameras. It was reall…”

“I would say the human node aspect is also very appealing because people can teleoperate it very easily. And so it's a data collection thing that is extremely helpful because people will be able to obviously very easily teleoperate it. I think that's usually ov…”

“When you're doing synthetic data generation, this is a problem, because you actually really want that entropy. You want the diversity and richness in your data set. Otherwise, you're getting collapsed data sets, and you can't see it when you look at any indivi…”

“And if we have this, for example, like Universal Translator, which I don't think is too far away, like you'll lose the ability to speak to people who don't speak English if you just put your stuff away.”

“Or like when you go to objects like in Disney all the objects are alive. And I think we are going to potentially come to that kind of a world where why can't I talk to things?”

“I think we'll probably end up with companies for, of LLMs. I think it's not unlikely to me that you have models of different capabilities specialized to various Unique domains. Maybe there's a programmer, et cetera. And it will actually start to resemble compa…”

“Obviously this will totally break down, especially in a society that's turning over so quickly. The people are going to come back to school A lot more frequently as the technology changes very, very quickly.”

“I didn't really take advantage of a lot of, like, email and calendar and all this other stuff, and I didn't give it access, because I'm still a little bit, like, suspicious, and it's still very new and rough around the edges, so I didn't want to give it, like,…”

“A lot of things have this property that, you know, very expensive to come up with, but very cheap to verify. And so in all those cases, things like folding at home or SETI at home or auto research at home will be good fits.”

“And now actually what we have is kind of like an equivalent of GitHub in the realm of software two point oh, and I think the hugging face is basically equivalent of GitHub in software two point oh, and there's also model Atlas, and you can visualize all the co…”

“Bursell, for example, is replacing every occurrence of click with the equivalent curl command that your LLM agent could take on your behalf.”

“We're training transformers in C at a pinch of C++.”

“So, PyTorch is really, really nice, and this is just some of the things that PyTorch offers. So, without PyTorch, we're kind of naked in the world, right?”

“It's a single file C. There's no dependencies whatsoever. It compiles instantly. It runs instantly. All the memory is just allocated in a single blob. So if you start stepping, there's no way you're gonna boom later. It's all preplanned. It's fully determinist…”

“All of this work that I described so far, I was on vacation and while I was jet-lagged in Maldives. So I, basically it's perfect because you wake up at one a.m., and there's nothing to do. So you write stuff like LLN.C.”

“We have almost a 50% NFU here on one node, which is quite good.”

“We will have Lama 3.1 training in Lama.c very, very soon.”

“It's only maybe like, I think, 3000 lines of code, basically C mostly.”