The Ledger, every show

“And the second was that reinforcement learning was going to be the biggest driver for LLM capabilities. Today, very obvious three years ago was not an opinion held or direction held at either OpenAI or Google or Anthropic or others.”

“When you look at the benchmarks and start using it, you'll realize that we are outperforming models two or three times their size.”

“Because while Google has to work with Broadcom to be able to bring TPUs into the world, Amazon is working with the fabs directly.”

“But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data.”

“What I would like to see and what I think is going to happen is that it's not just Google, Amazon, and Microsoft. It's going to be, you know, an open eye, an anthropic, a poolside, and a set of companies who are able to get that massive escape velocity needed …”

“Well, the first thing that I think is at this point, almost uncontested, but frankly, when I was building source, most people didn't think would happen is the fact that the majority, if not all developers in the next couple of years in the world will adopt som…”

“We don't think we're more than three years out to that point where a non-engineer, someone with no coding, you know, software development background is able to come and give one of those high level vague things of what they want built, right? Cause that's what…”

“A 118,000,000,008 B active model, which is not that large. It fits on a DGX spark and still runs at, you know, 3040 tokens a second on a spark is able to solve. Erdos three 97 independently. It's able to do complex programming tasks.”

“So at the beginning of Q four, end of Q three, so next year, we've got the first compute coming online. And then we're finalizing in Q one, 20, 27, the first you know, 250 megawatts, but already next summer, the next 250 megawatts starts construction. It's a s…”

“But if you think about the Malibu agent as a coding agent, for instance, right now, it sits at a level of like sweet bench verified, for instance, where Gemini two and a half pro was when it came out.”

“Usable code for training is what we refer to as about three trillion tokens. And if you look at kind of usable language in English on the internet for training, we're talking about anywhere between 10 and 15 trillion tokens. There's a massive amount of code th…”

“They built a 100,000 GPU, three, 32 K interconnected clusters in Tennessee in the span of months, and XAI showed up with Elon's strength, the ability to build physical infrastructure incredibly fast in the world.”

“Pretty much what we've seen consistently with every two-year generation of training from NVIDIA is about a two X, one and a half to two X, you know, performance increase. But training is about two X every two years.”

“We failed ultimately at the time, and it kind of was like biggest failure of my career, right? You blew twelve million dollars of investors' money, which was a lot back then.”

“They started years before ChatGPT.”