Poolside

“We're less than 70 researchers and other 35 engineers. And we're running, well, I haven't checked the latest count, but far more than 10,000, maybe 10 to 20,000 experiments a month.”

“Model building is ultimately 90% engineering. And I think we all know it in the industry, because if you look at words, every researcher spending their time, they're spending their time writing code, right? Looking at data and writing code.”

“The new medium started training, and it's much bigger than the last medium started training yesterday. So it's a 39 day pre-training run.”

“One of my favorite metrics about like Laguna S is that there was no on-call events, right? Like completely zero. And actually we haven't had a meaningful on-call event, like something to wake up for as far as I recall this entire year.”

“When we first wrote our first training code base completely from scratch, it wasn't a fork of any open source. It was just like, okay, let's build it from scratch.”

“My biggest wall clock bottleneck right now is RL time. And it's just because I can scale it up further because I can't add more GPUs to it because of that size constraint.”

“Laguna S was trained in FP-A, Only thing that in this run, I have to admit, that wasn't FPA. It was the all to all. In the new run we just started yesterday, the FPA was all to all.”

“I think it all just came down to one thing and I'll stop the monologue is the fact that I rather live in a world that has a hundred foundation model companies than a world that has five. Even if I was one of the five. And the smallest and most meaningful contr…”

“I think MCP and tools are stupid.”

“It's not very helpful to have a post training recipe for a smaller model and try to apply it to a bigger model. Yeah. It just, in all cases, you're gonna have to rethink most of the recipe. But recipe for post training for a bigger model applied to a smaller m…”

“We're like relatively small. We're a 10 K H 200 cluster company right now. We'll be scaling to a lot more soon, but and really a lot more.”

“We think focusing on coding and long horizon software tasks is a path towards AGI because it forces us to solve the hard problems.”

“Laguna access two that we launched. It was five weeks from the beginning of pre-training to launch. The model that we're going to talk about today was eight weeks from start to pre-training to launch.”

“When you look at the benchmarks and start using it, you'll realize that we are outperforming models two or three times their size.”

“The incredible Chinese lab has done an amazing job at sharing their research, and we have definitely take, like, been on the receiving end of taking advantage of that. So When you're on the receiving end of something coming to you, I think you also have kind o…”

“Once we realized that we treated data as immutable and code as always version, and you could always track and trace every experiment end to end perfectly, you could repeat everything perfectly, right? You have perfect reproducibility. I can still reproduce run…”

“We started as an American company. We have always been an American company. And early on, we made a very conscious decision. We said, we're not going to hire any researchers in the Bay Area. We're going to actually look for talent everywhere else in the world.”

“I actually think you can sum down. So I saw 90. Five percent of model building to just doing, you're just doing two things. You're improving data or you're improving compute efficiency.”

“You can't fine tune your way to success, right? Major capabilities emerge from training a base model made accurate and useful during fine tuning.”

“I think we ultimately only succeed if we scale our models as large as our competition. I do not, like, I think we should not put our head in the sand and say we're going to be king of open source small models. I think that's Frankly, it's a cop out.”

“Laguna S Laguna Small, 118, one, eight billion total parameters, eight B active. So very sparse. It's a scale up of the XS architecture. It's the kind of classic or quite classic these days, like three to one ratio of sliding window attention to global attenti…”

“One of the guys on our team who started as an engineer building our agents is a legit reinforcement learning researcher now making real progress. And that happened in the span of like six months that would have not been what I think most people assumed was pos…”