The Ledger, every show

“I would actually go as far as saying that intelligence is going to become a commodity.”

“A whole bunch of vertical software or vertical agents are probably gone.”

“And so to me, Elon has one full self-driving. I think it's inevitable that, that the most capable AI for full self-driving is coming out of Tesla because they've been gathering and building up this data set.”

“The research that still gets published openly, right, that doesn't get held back, that is most interesting, is all coming out of China in vast spades and majorities.”

“And so this is kind of our view and our, if you extend this a couple of years out, and this is where some people will probably have objection with us. We think that this will go so far. That we will kind of recycle all of the world's real data into higher qual…”

“And the second was that reinforcement learning was going to be the biggest driver for LLM capabilities. Today, very obvious three years ago was not an opinion held or direction held at either OpenAI or Google or Anthropic or others.”

“We've got a small window before models are Really impacting recursive self-improvement to a level where catching up otherwise might become unfeasible.”

“I have I would say a not commonly held opinion that reinforcement learning will move earlier and earlier into pre-training.”

“When you look at the benchmarks and start using it, you'll realize that we are outperforming models two or three times their size.”

“I think we will in 12 months not see a single system prompt that is stuffed with 20 or 30 or 40 tools anymore.”

“Intelligence is the most in life. You're going to be the world's most demanded commodity. It will more commoditize in margin and price.”

“The first paradigm of kind of pre-training of predicting the next token on the web was becoming sigmoidal and was slowing down in terms of the gains that it had.”

“I don't think anyone has any doubts anymore that we're now on track to reach human level capabilities and intelligence. And in that world, 29 trillion dollars of knowledge work rewrites itself, right?”

“I'm not yet convinced that gross margins in our industry will look like a SaaS company, like 80% plus. I think when we're talking about a commodity as intelligence, that we build value added services on top. Right. It's a foundation model company. On one hand,…”

“We think we'll push models to a level of reasoning and thought That will happen far earlier in their training than it does today.”

“And then the fourth stage of training over time increasingly will become learning, continuous learning from real world experiences of these agents. And so those are kind of the four stages that we think training will go to.”

“But we have a phrase at poolside, which is over time, everything collapses into the models.”

“I think we have a hard time holding the point of view that models will reach the same level of intelligence and capabilities that the world's most capable people in every field have. And when you take a step back, and don't take the next 12 month view, but jus…”

“At some point we are going to be in a world where across all sets of capabilities that we have as human beings, Machine intelligence is going to be as capable and if not more capable than us and surpass us. Now, our point of view is, is that that world is stil…”

“Our view of Poolside is that the first majorly economically viable capability that is going to close the gap between human intelligence and machine intelligence is software development.”

“Because while Google has to work with Broadcom to be able to bring TPUs into the world, Amazon is working with the fabs directly.”

“But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data.”

“Today, interconnecting more than 32,000 GPUs is extremely challenging.”

“If you want to become a hyperscaler that is able to put data centers all over the world with GPUs in it that is, that are going to allow you to serve these models to everyone, an infrastructure player, that's probably it. And that's probably just a starting po…”

“I think there's very few left to be acquired, to be very honest.”

“What I would like to see and what I think is going to happen is that it's not just Google, Amazon, and Microsoft. It's going to be, you know, an open eye, an anthropic, a poolside, and a set of companies who are able to get that massive escape velocity needed …”

“That it's very like that in our lifetime, neural networks will become capable of learning anything and everything that we are capable of as humans.”

“Well, the first thing that I think is at this point, almost uncontested, but frankly, when I was building source, most people didn't think would happen is the fact that the majority, if not all developers in the next couple of years in the world will adopt som…”

“We don't think we're more than three years out to that point where a non-engineer, someone with no coding, you know, software development background is able to come and give one of those high level vague things of what they want built, right? Cause that's what…”

“We've decided to scale up rev nets, reversible layers. So, so to the best of our knowledge, I think we're training the world's largest models using reversible layers that, that we've seen at least that nothing else that we're aware of publicly.”

“We went through about 3200 or 3200 people that we ended up putting on the list. And we had this location column, like think genuinely Google spreadsheet, you know, like where's someone based. And at some point we looked at all these locations and we're like, w…”

“Laguna access two that we launched. It was five weeks from the beginning of pre-training to launch. The model that we're going to talk about today was eight weeks from start to pre-training to launch.”

“A 118,000,000,008 B active model, which is not that large. It fits on a DGX spark and still runs at, you know, 3040 tokens a second on a spark is able to solve. Erdos three 97 independently. It's able to do complex programming tasks.”

“The difference of a model you could train on hoppers versus GB 300 is the difference in a trillion parameter model and a five or six trillion parameter model.”

“I don't think it's a winner takes all market, but it does seem to be that it's a small number of companies who are able to get there.”

“To energize, 250 megawatts is about half a billion dollars worth of physical infrastructure. So in the case of gas turbines, it's gas turbines, but there's different paths to it. Building a data center, like the powered shell that brings together all of the eq…”

“So at the beginning of Q four, end of Q three, so next year, we've got the first compute coming online. And then we're finalizing in Q one, 20, 27, the first you know, 250 megawatts, but already next summer, the next 250 megawatts starts construction. It's a s…”

“Renewables only, in combination with the battery capacity that you would need to power a data center if you want renewable only, it's just not economically viable yet.”

“But if you think about the Malibu agent as a coding agent, for instance, right now, it sits at a level of like sweet bench verified, for instance, where Gemini two and a half pro was when it came out.”

“Usable code for training is what we refer to as about three trillion tokens. And if you look at kind of usable language in English on the internet for training, we're talking about anywhere between 10 and 15 trillion tokens. There's a massive amount of code th…”

“We put it in an environment, say it's an environment with a 130,000 real world code bases, several orders of magnitude, the largest environment in the world.”

“They built a 100,000 GPU, three, 32 K interconnected clusters in Tennessee in the span of months, and XAI showed up with Elon's strength, the ability to build physical infrastructure incredibly fast in the world.”

“The world is finding a balance and the balance that I'd like to see that the world is, and I think it's moving towards it, is to regulate the end user application of AI.”

“We're less than 70 researchers and other 35 engineers. And we're running, well, I haven't checked the latest count, but far more than 10,000, maybe 10 to 20,000 experiments a month.”

“One of my favorite metrics about like Laguna S is that there was no on-call events, right? Like completely zero. And actually we haven't had a meaningful on-call event, like something to wake up for as far as I recall this entire year.”

“What was particularly interesting to learn about the infrastructure space over time is that it's an incredibly small world. Everybody knows each other. It makes tech feel like a big world, right? The construction of data centers at scale has really only been d…”

“And I think what we'll see in the next couple of years is a lot of model companies ourselves as well. We go into these enterprises. We find high value use cases. We use our agents to get to 60 or 70% You know, of quality, and then use additional reinforcement …”

“Foundation models today can't do it, and we don't yet have a path In them successfully doing so in a way that, like, really improves it.”