The Ledger, every show

“Because yeah, getting acquired would be really limiting. It would be this admission of failure and jumping ship because you can't make it on your own anymore.”

“Like I used to work at a bunch of the big tech companies, and I always felt that we could fire 90% of people and we would move faster because the best people wouldn't have all these distractions.”

“A lot of researchers, they'll tell us that their VPs make them focus on increasing their rank on LMSYS. And so I've had researchers explicitly tell me that they're okay with making their models worse. Add factuality works at following instructions as long as i…”

“No, I mean, I definitely wouldn't sell for thirty billion or even a hundred billion.”

“A lot of them tell us that even a thousand or a couple of thousand pieces of really high quality human data that we generated for them, it's actually been worth more than ten million pieces of synthetic data.”

“Yeah, so we hit over a billion of revenue last year with under a hundred people.”

“And I think we're going to see companies with even crazier ratios, like a hundred billion per employee in the next few years. AI is just going to get better and better and make things more efficient. So that ratio just becomes inevitable.”

“Sometimes, yeah, these benchmarks, they accidentally leak in certain ways, or the frontier labs will tweak the way they evaluate their models on these benchmarks. Like they'll tweak their system prompt. Or they'll tweak the number of times they run their model…”

“The easiest way to climb Alamarina, it's adding crazy boating. It's doubling the number of emojis. It's tripling the length of your model responses. Even if your model starts hallucinating and getting the answer completely wrong.”

“I definitely think that Vibe coding is overhyped. I think people don't realize, How much it's going to make your systems unmaintainable in the long term and decently dump this code into your code bases.”

“Like, if you talk to a bunch of YC founders or whoever it is, like, what is their goal? It really is to tell all their friends that they raised ten million dollars and show their parents they got a headline on TechCrunch. Like, that is their goal.”

“One of the things I think is a giant plague on AI is Elimsis, Elimarena.”

“I think a lot of the other companies in our space, they're just not technology companies at the end of the day. They are either body shops or they are body shops masquerading as technology companies.”

“Some people are simply two to three times better, like two to three times faster than anybody else, right? They just code faster. There are some people who simply have two to three times more better ideas. There are people who simply work two to three times as…”

“I think one of the things that's always driven me crazy about Silicon Valley is that it really is just a status game for most people. Like, People are just raising for the sake of raising. Their goal isn't to build some great product that solves an idea that t…”

“There are some companies in this space who will simply think of it as a pure supply problem, and they don't give any consideration to the technology, like both the technology, the underlying technology, like, how do you identify these people? How do you make s…”

“I mean, so I would say I'm pretty sure that a lot of these other companies, they are Like at the end of the day, people want high quality data and they don't want to be working with body shops. And so. I think we've seen, like, a massive wave interest because,…”

“Like if you think of all the PhDs, even at Google or Meta or Microsoft, we have way more than all of them combined doing work for us in a single day.”

“So LM Arena is this popular leaderboard of LM models, and it's basically the equivalent of clickbait.”

“So I think a lot of people think AI safety is overblown, but I think they ignore the paperclip maximizer problem where you have AI models that are accidentally trained towards the wrong objectives, even though this is a big problem that all the models face tod…”

“Don't think so yet. I can actually see big, new, even more powerful model developers appearing in the next few years.”

“One of the things I often think about is that there's a, it's almost like there's an art to post training. It's not purely a science. Like when you were deciding what kind of model you're trying to create and what it's good at. There's this notion of taste and…”

“I don't trust the benchmarks at all. And I think that's for two reasons. So one is, I think a lot of people don't realize, even researchers within the community, they don't realize that the benchmarks themselves are often honestly just wrong. Like they have wr…”

“In my head, I probably bet that within the next one or two years, yeah, the models are going to automate 80% of, you know, the average L six software engineer's job. But it's going to take another few years, do you move to 90%, and another few years to 99%, an…”

“I would say I've always been very, very impressed by Anthropic. Like, I think Anthropic takes a very principled view about what they do and don't care about. And how they want their models to behave in a way that feels a lot more principle to me.”

“I'm in a camp where I do believe that something new will be needed.”

“I think one of the things that's going to happen in the next few years is that the models are actually going to become increasingly differentiated because of the personalities and behaviors That the different labs have and the kind of objective functions that …”

“We are kind of like the biggest human data player in this space.”

“A lot of other companies in this space, they are essentially just body shops. What they are delivering is not data. They are literally just delivering warm bodies to companies. And so what that means is like at the end of the day, they don't have any technolog…”

“It's been beneficial because yeah, there were still some legacy teams using scale. Like they just didn't know about us because we were still pretty under the radar.”

“Like one of the things that we found is that the easiest way to improve your rank on album arena is the ability to make your model response longer.”

“Yeah, so I think the biggest lesson for me was that you can build a completely different kind of company with 10% of the resources and 10% of the people, but you're still moving 10 times faster and building a 10 times better product.”

“Like, it's almost like a negative sign if you're having a one-on-one weekly meeting, because it means that you just don't know what's going on with these people. You're not, you're like almost waiting for your weekly meeting to raise, raise interesting questio…”

“I mean, I absolutely believe that that company was this one day. Like you think about it, like I've always believed in 10 X engineers, even a hundred X engineers. And already you have a lot of these single person startups that are already doing ten million rev…”

“If you don't have to spend that time on the drudgery, but you just have like these endless ideas that are just bouncing around your head and AI just helps you put them to paper. Then I do think it kind of disproportionately favors people who are already like t…”

“I think half of the people who graduate with a CS degree, they can't even code.”

“As opposed to feeling like I needed to go out and hire 10 engineers in order to build a product, instead of feeling like I needed to go out and fundraise, you know, 10, 20, thirty million dollars in order to hire, you know, more people. Again, I just wanted to…”

“So it's interesting because I think it was an open secret where a lot of top researchers already knew who we were. And they already knew that we were the biggest and the best in the space, even though we've been pretty under the radar. And so most people were …”

“So I would definitely rank data quality first, followed by compute, followed by the IRLs.”

“And so one of the, I mean, one of the things that we've learned is that the easiest way to improve in this arena is simply to make your model responses a lot longer.”

“Synthetic data, it's made models good at synthetic problems, not, not real ones.”

“You always need this external value system as a kind of safeguard to make sure that the models are working properly just because the models themselves have such a different set of way of thinking.”

“One area where I'm not great is I'm really bad at understanding financials. So sometimes people around the company, they'll try to tell me, Like, hey, do you, have you been paying attention to our revenue numbers? Have you been paying attention to our costs? H…”

“So I think it would be twenty-twenty-eight if you're talking about automating a job at the average engineer, and then twenty-thirty-eight if we're talking about curing cancer.”

“How I see a world where there actually will be multiple, multiple, Frontier AI companies, Frontier AGIs, just because every one of them will be able to go in a different direction. Like, you see it already, you see it today playing out already with the differe…”

“I think most people don't understand what quality even means in this space. They think you can just throw bodies at a problem and get good data, and that's completely wrong.”

“The way it works is we essentially gather thousands of signals about everything that you're doing when you're working on a platform. So we are looking at your keyboard strokes. We are looking how fast you answer things. We are using reviews. We are using code …”

“Silicon Valley loves to score on Wall Street for focusing on money. But honestly, most of the Silicon Valley is chasing the same thing.”