The Ledger
Every statement that passed quotation and attribution checks. Mix any filter with any other: certainty 1/5, debate potential 5/5, or both at once.
why aren't all 60 resolved? a statement only gets an assessment when the public record can support or contradict it. opinions and what-ifs never can, and 1 checkable ones are still open, waiting for their date. predictions held up or didn't; assertions are supported or contradicted. on every card: ▮▮▮▮▮ certainty · ▮▮▮▮▮ debate potential. speakers are clickable
Karpathy: Tesla is ahead of Waymo because software is easier than hardware
“I think, personally, Tesla is ahead of Waymo, and I know it doesn't look like that, but I'm still very bullish on Tesla and its self-driving program. I think that Tesla has a software problem, and I think Waymo has a hardware problem, is the way I put it, and …”
Karpathy: Coding agent failures are user skill issues, not capability limits
“Even if they don't work, I think to a large extent, you feel like it's a skill issue. It's not that the capability is not there. It's that you just haven't found a way to string it together of what's available. Like I just don't, I didn't give good enough inst…”
Karpathy: Claude feels like a teammate, whereas Codex coding agent is dry
“I actually think Claude has a pretty good personality. It feels like a teammate and it's excited with you, et cetera. I would say for example, Codex is a lot more dry which is kind of interesting because in Chashi PT, Codex is like a lot more upbeat and highly…”
Karpathy: Software industry must reconfigure for agent customers instead of humans
“So I think the industry just has to reconfigure in so many ways that it's like the customer is not the human anymore. It's like agents who are acting on behalf of humans. And this refactoring will be, will probably be substantial in certain sense.”
Karpathy: Autonomous agent tool creation will be trivial within three years
“I kind of feel like this kind of stuff that I just talked about, this should be free, like in a year or two or three. There's no bi-coding involved. This is trivial. This is table stakes. This is like any AI, even the open source models, et cetera, can like do…”
Karpathy: Maximizing AI leverage requires removing humans from prompting loops entirely
“To get the most out of the tools that have become available now, you have to remove yourself as the bottleneck. You can't be there to prompt the next thing. You're, you need to take yourself outside
you have to arrange things such that they're completely auto…”
Karpathy: All frontier AI labs are pursuing recursive LLM self-improvement
“What I'm more interested in is, like, this idea of recursive self-improvement and to what extent you can actually have LLMs improving LLMs, because I think all the Frontier Labs, this is, like,
The thing for obvious reasons.
And they're all trying to recursive…”
Karpathy: A research organization can be defined as a set of markdown files
“A research organization is a set of markdown files that describe all the roles and how the whole thing connects.”
Karpathy: Smarter AI models do not automatically gain broad societal capabilities for free
“The story is that we're getting a lot of the intelligence and capabilities in all the domains of society, like, for free as we get better and better models, and it's not, like, exactly fundamentally what's going on, and there's some blind spots, and some thing…”
Karpathy: AI models will speciate into specialized smaller domain experts
“I do think that we will, we, I do think we should expect more speciation in the intelligences.”
Karpathy: Swarms of internet agents could outpace frontier AI labs
“A swarm of agents on the internet could collaborate to improve LLMs and could potentially even, like, run circles around Frontier Labs. Like, who knows, you know? Yeah, like, maybe that's even possible. Like, Frontier Labs have a huge amount of trusted compute…”
Karpathy: Cheaper software via AI will trigger Jevons paradox increasing demand
“So if the barrier comes down, then actually you have the Jevons paradox, which is, like, you know, you actually, the demand for software actually goes up. It's cheaper and there's more”
Karpathy: Frontier AI researchers are actively automating themselves out of jobs
“Even with other research, like, OpenAI or, you know Anthropic or these other labs, like, they're employing, what, like, a thousand something researchers, right? These researchers are basically, like, glorified auto, like, you know. They're, like, automating th…”
Karpathy: Frontier lab researchers face pressure over what they can say
“Like, if you're inside one of the frontier labs, like, there are certain things that you can't say and conversely, there are certain things that the organization wants you to say, and, you know, they're not gonna twist your arm, but you feel the pressure of, l…”
Karpathy: AI researchers lose technical judgment after leaving frontier labs
“And I think if you're outside of that frontier lab your judgment fundamentally will start to drift because you're not part of the, you know, what's coming down the line.”
Karpathy: Open source trails closed frontier AI models by 6 to 8 months
“So there may be, they're behind by like, what is the latest, maybe like eight months, six months, eight months kind of thing right now.”
Karpathy: Solely relying on closed AI models creates systemic risk
“I don't actually think it's like structurally, I think there's some systemic risk attached to just having intelligences that are closed, and that's like, that's it. And I think that that's a, you know, centralization has a very poor track record in my view”
Karpathy: Educators will soon teach AI agents instead of human students
“I feel like there's gonna be less of like explaining things directly to people. And it's gonna be more of just like, does the agent get it? And if the agent gets it, they'll do the explanation.”
Karpathy: Software documentation will shift from human HTML to agent markdown
“Instead of HTML documents for humans, you have markdown documents for agents, because if agents get it, then they can just explain all the different parts of it. So it's this redirection through agents, you know and that's like, so I think we're going to see a…”
Karpathy: Cognitive core models will likely shrink to one billion parameters
“I think even a billion. Billions of ISIS. We'll probably get to that point and the models can be very, very small.”
Tesla Uses LiDAR During Training for Its Vision-Only Autonomous Stack
“Tesla actually does use a lot of expensive sensors. They just do it at training time. So there are a bunch of cars that drive around with LIDARs. They do a bunch of stuff that like doesn't scale and they have extra sensors, et cetera, and they do mapping and a…”
Karpathy: Tesla Autonomous Stack Will Be Pure End-to-End Neural Net in 10 Years
“And I do suspect that The end-to-end systems for Tesla in, like, say, 10 years, it is just a neural net. I mean, the videos stream into a neural net and commands come out.”
Karpathy: Tesla is fundamentally a robotics-at-scale company, not a carmaker
“Basically everything transfers and I don't think people appreciate it. Cars are robots and Tesla, I don't think is a car company. I think this is misleading. This is a robotics company, robotics at scale company, because I would say at scale is also like a who…”
Karpathy: No single technical bottleneck is blocking robotics, just data grunt work
“I don't know that there's, like, any individual impediments that I'm, like, really familiar with. I just think it's a lot of grunt work. A lot of, like, the tools are available. Transformers are this beautiful, like, blob of tissue. You can just get just arbit…”
Karpathy: Clean AI scaling laws are a property of transformers, not LSTMs
“When people talk about the scaling loss in neural networks, the scaling laws are actually a to a large extent of a property of the transformer. Before the transformer, people were playing with LSTMs and stacking them, etc. You don't actually get like clean sca…”
Karpathy: Model architecture is no longer the fundamental bottleneck in AI
“I don't think that the neural network architecture is like holding us back fundamentally anymore. It's like not the bottleneck, whereas I think in the previous, before Transformer, it was a bottleneck, but now it's not the bottleneck. So now we're talking a lo…”
Karpathy: One billion human thought trajectories would roughly achieve AGI
“The trajectories in your brain as you're doing problem solving. If we had a billion of that, like AGI is here, roughly speaking. I mean, to a very large extent.”
Karpathy: AI will not run out of training data due to synthetic data
“So I think basically synthetic data is absolutely the future. We're not going to run out of data, is my impression. I just think you have to be careful.”
Karpathy: Transformers are a more efficient system than the human brain
“I think transformers are actually better than the human brain in a bunch of ways. I think they're actually a lot more efficient system. And the reason they don't work as good as the human brain is mostly data issue, roughly speaking, is the first order approxi…”
Karpathy: A 1-billion parameter model will suffice as a cognitive core
“I think even a 1,000,000,001, billion suffices. We'll probably get to that point, and the models can be very, very small. And I think the reason they can be very small is fundamentally, I think, just, like, distillation works.”
Karpathy: Most activity in AI aims to displace workers
“I think there's a lot of activity in, like, AI, and I think most of it is to kind of, like, replace or displace people, I would say.”
Karpathy: AI cannot design courses yet, but can act as TA
“Currently at the current AI capability, I don't think the models are good enough to create a good course. But I think they're good to become the front end to the student and interpret the course to them.”
Karpathy: Math, physics, and CS are the best foundation for children's thinking skills
“And the correct answer is mostly like I would say like math, physics, CS kind of disciplines. And the reason I say that is because I think it helps for just thinking skills. It's just like the best thinking skill core. Is, is my opinion.”
Karpathy: Has not typed a line of code since December
“I don't think I've typed like a line of code. Probably since December, basically which is like an extremely large change.”
Karpathy: Parallel AI agents elevate software engineering to macro actions
“It's just like you can move in much larger macro actions. It's not just like, here's a line of code, here's a new function. It's like, here's a new functionality. And delegate it to agent one. Here's a new functionality that's not going to interfere with the o…”
Karpathy: AutoResearch discovered hyperparameter tunings he missed after manual optimization
“And then I let our research go for like overnight
and it came back with like tunings that I didn't see.
And yeah, I did forget like the weight decay on the value embeddings and my atom betas were not sufficiently tuned and these things jointly interact.”
Karpathy: AutoResearch is limited strictly to domains with easily evaluatable objective metrics
“This is extremely well suited to anything that has objective metrics that are easy to evaluate. So for example, like writing kernels for more efficient CUDA, you know, code for various parts of a model, etc., are the perfect fit. Because you have inefficient c…”
Karpathy: Digital AI automation will outpace physical automation at speed of light
“Flipping, flipping bits and the ability to copy paste digital information is like, makes everything a million times faster than accelerating matter, you know? So so energetically, I just think we're going to see a huge amount of activity in digital space, huge…”
Karpathy: AI innovation will move from digital unhobbling to physical interfaces
“First, there's gonna be a huge amount of unhobbling, and I think there's a huge amount of work there. Then, actually, it's going to move to, like, the interfaces between physical and digital. So, and that's, like, sensors of, like, seeing the world and actuato…”
Karpathy: Core LLM training algorithm requires only 200 lines of Python
“Training neural nets and LLMs specifically is a huge amount of code, but all of that code is actually complexity from efficiency. It's just because you need it to go fast. If you don't need it to go fast and you just care about the algorithm, then that algorit…”
Karpathy: Humanoid robots will scale internally first, then B2B, then consumer
“I think B to C should be the right Start point, because I don't think we can have a robot, like, crush grandma, is how I put it, sort of. I think it's like too much legal liability. It's just like, I don't think that's the right approach... So I think the best…”
Karpathy: General robot platforms uniquely enable cross-task transfer learning in AI
“And then I would say also one last dimension of it is you benefit a ton from like the transfer learning between the different tasks. And in AI, you really want a single neural nut that is multitasking, doing lots of things that's very getting all the intellige…”
Karpathy: RoPE is the only major transformer architecture change in five years
“The transformer hasn't changed that much. You know, we've added the rope positional and the rope relative positional encodings. That's like the major change. Everything else doesn't really matter too much. It's like plus three percent on a small few things. Bu…”
Karpathy: In AI, the demo is near but the product is far
“That's the thing with AI. I feel like a lot of them, a lot of these capabilities are just kind of like prompt away. So you always get like demos, but like, do you actually get a product? You know what I mean? So so in this sense, I would say the demo is near, …”
Karpathy: Academic lab lineages act as gatekeeping and AI should dismantle them
“I don't actually want to live in a world where lineage like matters too much, right? So I'm hoping that AI can help you destroy that structure a little bit. It feels like kind of gatekeeping by some finite scarce resource, which is like, oh, there's a finite n…”
Karpathy: True learning requires effortful exertion like the gym, not passive entertainment
“It takes effort, but it's effortful, but it's also kind of fun. And you also have a payoff of like, You feel good about yourself in various ways, right? And I think education is basically equivalent to that. So that's what I mean when I say education should no…”
Karpathy: Engineering productivity shifted from GPU FLOPS to agent token throughput
“You would feel nervous when your GPUs are not running. Like you have GPU capability and you're not maximizing the available flops to you. But now it's not about flops, it's about tokens. So what is your token throughput and what token throughput do you command…”
Karpathy: Autonomous agent 'Dobby' manages all home systems and security cameras
“So it controls all of my lights, my HVAC, my shades the pool and the spa, and also my security system. So I have a camera pointed outside of the house, and anytime someone rolls in, I have a Quinn a Quinn model that looks at the videos.”