neural nets

“We learned that scale really does work. And in that case, it wasn't scaling, you know, training and neural nets, it was scaling search.”

“Like you're about to experience, like the analysis of these neural nets is going to look like systems biology, right? It's going to be like, go in and like, try to back, figure out a thing that you didn't design, my friends.”

“I'm still waiting for the statistical learning and maybe some of the convex optimization stuff to reemerge. I wouldn't count it entirely out yet. I feel like the pendulum always swings back the other way. It's swung away from statistical learning and convex op…”

“We're doing that fully autonomous end to end on our robots now, all bipedal. And the second is we're doing kind of full consumer level manipulation and like speech to speech reasoning. So we're able to Talk with the robot. It's able to understand what we're sa…”

“Actually, like, end-to-end driving, when you're just imitating humans and so on, you have very few bits of supervision to train a massive neural net. And it's too Too few bits of signal to train so many billions of parameters. And so these intermediate represe…”

“And I do suspect that The end-to-end systems for Tesla in, like, say, 10 years, it is just a neural net. I mean, the videos stream into a neural net and commands come out.”

“Training neural nets and LLMs specifically is a huge amount of code, but all of that code is actually complexity from efficiency. It's just because you need it to go fast. If you don't need it to go fast and you just care about the algorithm, then that algorit…”