The Ledger, every show

“A lot of researchers, they'll tell us that their VPs make them focus on increasing their rank on LMSYS. And so I've had researchers explicitly tell me that they're okay with making their models worse. Add factuality works at following instructions as long as i…”

“No, I mean, I definitely wouldn't sell for thirty billion or even a hundred billion.”

“A lot of them tell us that even a thousand or a couple of thousand pieces of really high quality human data that we generated for them, it's actually been worth more than ten million pieces of synthetic data.”

“Yeah, so we hit over a billion of revenue last year with under a hundred people.”

“And I think we're going to see companies with even crazier ratios, like a hundred billion per employee in the next few years. AI is just going to get better and better and make things more efficient. So that ratio just becomes inevitable.”

“Sometimes, yeah, these benchmarks, they accidentally leak in certain ways, or the frontier labs will tweak the way they evaluate their models on these benchmarks. Like they'll tweak their system prompt. Or they'll tweak the number of times they run their model…”

“The easiest way to climb Alamarina, it's adding crazy boating. It's doubling the number of emojis. It's tripling the length of your model responses. Even if your model starts hallucinating and getting the answer completely wrong.”

“Like if you think of all the PhDs, even at Google or Meta or Microsoft, we have way more than all of them combined doing work for us in a single day.”

“Don't think so yet. I can actually see big, new, even more powerful model developers appearing in the next few years.”

“In my head, I probably bet that within the next one or two years, yeah, the models are going to automate 80% of, you know, the average L six software engineer's job. But it's going to take another few years, do you move to 90%, and another few years to 99%, an…”

“I'm in a camp where I do believe that something new will be needed.”

“I think one of the things that's going to happen in the next few years is that the models are actually going to become increasingly differentiated because of the personalities and behaviors That the different labs have and the kind of objective functions that …”

“We are kind of like the biggest human data player in this space.”

“Like one of the things that we found is that the easiest way to improve your rank on album arena is the ability to make your model response longer.”

“I mean, I absolutely believe that that company was this one day. Like you think about it, like I've always believed in 10 X engineers, even a hundred X engineers. And already you have a lot of these single person startups that are already doing ten million rev…”

“So it's interesting because I think it was an open secret where a lot of top researchers already knew who we were. And they already knew that we were the biggest and the best in the space, even though we've been pretty under the radar. And so most people were …”

“One area where I'm not great is I'm really bad at understanding financials. So sometimes people around the company, they'll try to tell me, Like, hey, do you, have you been paying attention to our revenue numbers? Have you been paying attention to our costs? H…”

“So I think it would be twenty-twenty-eight if you're talking about automating a job at the average engineer, and then twenty-thirty-eight if we're talking about curing cancer.”

“How I see a world where there actually will be multiple, multiple, Frontier AI companies, Frontier AGIs, just because every one of them will be able to go in a different direction. Like, you see it already, you see it today playing out already with the differe…”

“So I actually see more and more frontier models opening up over time because I actually don't think that the models will be commodities.”

“And already you have a lot of these single person startups that are already doing ten million revenue.”

“And we're about a hundred, a little over a hundred people.”

“Like, I don't think our environments alone will suffice just because, I mean, it depends on how you think about our environments, but oftentimes these are very, very rich trajectories are very, very long. And so it's almost like inconceivable that a single rew…”

“Like, I think it is really interesting in that a lot of the, like for obvious reasons, a lot of the frontier labs, they're just not publishing anymore.”

“Historically, a lot of ML engineers, they kind of just don't take the time to look at the data.”

“We were very lucky to be profitable from month one.”

“Things definitely hit an excellent point with ChatGPT because I think people just saw how Incredibly valuable human data and RHF was. So definitely chat CPT was an inflection point for us, but even before that we were, we had very strong growth.”