The Ledger, every show
Every statement that passed quotation and attribution checks, across all 44 shows. Pick shows below, then mix any filter with any other.
shows 
every show 44 of 44
Karpathy: Software industry must reconfigure for agent customers instead of humans
“So I think the industry just has to reconfigure in so many ways that it's like the customer is not the human anymore. It's like agents who are acting on behalf of humans. And this refactoring will be, will probably be substantial in certain sense.”
Karpathy: Autonomous agent tool creation will be trivial within three years
“I kind of feel like this kind of stuff that I just talked about, this should be free, like in a year or two or three. There's no bi-coding involved. This is trivial. This is table stakes. This is like any AI, even the open source models, et cetera, can like do…”
Karpathy: All frontier AI labs are pursuing recursive LLM self-improvement
“What I'm more interested in is, like, this idea of recursive self-improvement and to what extent you can actually have LLMs improving LLMs, because I think all the Frontier Labs, this is, like,
The thing for obvious reasons.
And they're all trying to recursive…”
Karpathy: AI models will speciate into specialized smaller domain experts
“I do think that we will, we, I do think we should expect more speciation in the intelligences.”
Karpathy: Swarms of internet agents could outpace frontier AI labs
“A swarm of agents on the internet could collaborate to improve LLMs and could potentially even, like, run circles around Frontier Labs. Like, who knows, you know? Yeah, like, maybe that's even possible. Like, Frontier Labs have a huge amount of trusted compute…”
Karpathy: Open source trails closed frontier AI models by 6 to 8 months
“So there may be, they're behind by like, what is the latest, maybe like eight months, six months, eight months kind of thing right now.”
Karpathy: Educators will soon teach AI agents instead of human students
“I feel like there's gonna be less of like explaining things directly to people. And it's gonna be more of just like, does the agent get it? And if the agent gets it, they'll do the explanation.”
Karpathy: Software documentation will shift from human HTML to agent markdown
“Instead of HTML documents for humans, you have markdown documents for agents, because if agents get it, then they can just explain all the different parts of it. So it's this redirection through agents, you know and that's like, so I think we're going to see a…”
Waymo autonomous vehicles still rely heavily on human teleoperation
“Like, you may see Waymo's going around, and they look driverless, but, you know, there's still a lot of teleoperation and a lot of human in the loop of a lot of this driving.”
Karpathy: Cognitive core models will likely shrink to one billion parameters
“I think even a billion. Billions of ISIS. We'll probably get to that point and the models can be very, very small.”
Karpathy predicts LLMs will act as compilers generating bare-metal CUDA code
“If LLINs are about to become much better at coding over time, then I think you can expect that the LLIN could actually do this for any custom application over time. And so the LLINs could act as a kind of compiler What you're interested in, they're gonna do al…”
Tesla Uses LiDAR During Training for Its Vision-Only Autonomous Stack
“Tesla actually does use a lot of expensive sensors. They just do it at training time. So there are a bunch of cars that drive around with LIDARs. They do a bunch of stuff that like doesn't scale and they have extra sensors, et cetera, and they do mapping and a…”
Karpathy: Tesla Autonomous Stack Will Be Pure End-to-End Neural Net in 10 Years
“And I do suspect that The end-to-end systems for Tesla in, like, say, 10 years, it is just a neural net. I mean, the videos stream into a neural net and commands come out.”
Karpathy: One billion human thought trajectories would roughly achieve AGI
“The trajectories in your brain as you're doing problem solving. If we had a billion of that, like AGI is here, roughly speaking. I mean, to a very large extent.”
Karpathy: AI will not run out of training data due to synthetic data
“So I think basically synthetic data is absolutely the future. We're not going to run out of data, is my impression. I just think you have to be careful.”
Karpathy: A 1-billion parameter model will suffice as a cognitive core
“I think even a 1,000,000,001, billion suffices. We'll probably get to that point, and the models can be very, very small. And I think the reason they can be very small is fundamentally, I think, just, like, distillation works.”
Karpathy: AutoResearch discovered hyperparameter tunings he missed after manual optimization
“And then I let our research go for like overnight
and it came back with like tunings that I didn't see.
And yeah, I did forget like the weight decay on the value embeddings and my atom betas were not sufficiently tuned and these things jointly interact.”
Karpathy: AI innovation will move from digital unhobbling to physical interfaces
“First, there's gonna be a huge amount of unhobbling, and I think there's a huge amount of work there. Then, actually, it's going to move to, like, the interfaces between physical and digital. So, and that's, like, sensors of, like, seeing the world and actuato…”
Mac Minis are uniquely suited for batch-one local LLM inference
“Mac minis, for example, are a very good fit for some of the LLMs, because it's all, if you're doing batch one inference, this is all super memory bound, so this actually works.”
Karpathy: llm.c trains GPT-2 on one H100 node in 24 hours for $600
“You can train it on a single node of H-one-hundreds in about 24 hours, and that costs roughly 600 dollars.”
Karpathy: llm.c was 20% faster and used 30% less memory than PyTorch
“At the time of that post, we were using, in LL and that's in 30% less memory, and we were 20% faster in training, just the truth.”
Karpathy: Humanoid robots will scale internally first, then B2B, then consumer
“I think B to C should be the right Start point, because I don't think we can have a robot, like, crush grandma, is how I put it, sort of. I think it's like too much legal liability. It's just like, I don't think that's the right approach... So I think the best…”
Karpathy: RoPE is the only major transformer architecture change in five years
“The transformer hasn't changed that much. You know, we've added the rope positional and the rope relative positional encodings. That's like the major change. Everything else doesn't really matter too much. It's like plus three percent on a small few things. Bu…”
Karpathy: Autonomous training loops will overfit metrics via Goodhart's Law
“I do think that if you had an autonomous loop over those metrics, there's gonna be a lot of like good harding going on where the system will like overfit to those metrics.”
Tesla Autopilot neural networks systematically replaced explicit C++ code
“As we made the autopilot better, basically the neural network grew in capability and size, and in addition to that, all the C++ code was being deleted and kind of like was And a lot of the kind of capabilities and functionality that was originally written in o…”
Vibe coding will become a gateway drug to software development
“I think this will end up being like a gateway drug to software development.”
Karpathy: Early Tesla Optimus ran car neural networks and vehicle computers
“In terms of the transfer from cars to humanoids, it was not, not that much work at all. And in fact, like the early versions of Optimus the robot it thought it was a car, like, because it had the exact same computer. It had the exact same cameras. It was reall…”
Karpathy: Universal translator AI is near and will make users tech-dependent
“And if we have this, for example, like Universal Translator, which I don't think is too far away, like you'll lose the ability to speak to people who don't speak English if you just put your stuff away.”
Karpathy: The world will reach ubiquitous conversational AI in everyday objects
“Or like when you go to objects like in Disney all the objects are alive. And I think we are going to potentially come to that kind of a world where why can't I talk to things?”
Karpathy: AI workflows will evolve into swarms resembling corporate hierarchies
“I think we'll probably end up with companies for, of LLMs. I think it's not unlikely to me that you have models of different capabilities specialized to various Unique domains. Maybe there's a programmer, et cetera. And it will actually start to resemble compa…”
Karpathy: Linear schooling will break down into lifelong education cycles
“Obviously this will totally break down, especially in a society that's turning over so quickly. The people are going to come back to school A lot more frequently as the technology changes very, very quickly.”
Vercel replaces click UI instructions with cURL commands for AI agents
“Bursell, for example, is replacing every occurrence of click with the equivalent curl command that your LLM agent could take on your behalf.”
Karpathy: llm.c avoids runtime crashes by pre-allocating memory statically without dependencies
“It's a single file C. There's no dependencies whatsoever. It compiles instantly. It runs instantly. All the memory is just allocated in a single blob. So if you start stepping, there's no way you're gonna boom later. It's all preplanned. It's fully determinist…”
Karpathy: llm.c achieves nearly 50% MFU on single-node GPT-2 training
“We have almost a 50% NFU here on one node, which is quite good.”