The Exchanges

Q But before we, I do want to touch on kind of the working hours that you mentioned that, but everyone poses synthetic data as a big threat. And what happens to your business when we have synthetic data, That is obviously created, uh, automatically and labeled automatically. How do you think about the role of human labeled data in a world of predominantly synthetic data? What's your thoughts there?

A So I think synthetic data is actually really useful in some places, but I think people overestimate what it can do. So I'll, I'll give a couple examples. So right now there are a bunch of models that have been trained really heavily on synthetic data. But like I mentioned earlier, it means that they're only good at very academic, homework style, benchmark style problems. They're actually terrible at real world use cases. So yeah, synthetic data, it's made models good at synthetic problems, not, not real ones. And we actually hear from a lot of companies who tell us they spent the past year training their models on synthetic data, but if only now just realize all the problems that's caused. And so they've spent actually months throwing a lot of it out. Like a lot of them tell us that even a thousand or a couple of thousand pieces of really high quality human data that we generated for them, it's actually been worth more than ten million pieces of synthetic data. And so a lot of the work that we do, it's simply cleaning up all this synthetic data. And if you think about like why this happens, it essentially is because the models collapse on this very, very narrow scope. Of, um, of, like, similarity that the synthetic data creates, and so it just doesn't give the models the kind of the diversity and generalizability that they need. And then one other point is that there's also thi…

AI assessment note: “high quality human data that we generated for them, it's actually been worth more”

Q Okay, so you realize this data problem in twenty-twenty, you leave Twitter. What happens then? You go heads down into product build for several months. You go about recruiting the first team members. Can you just take me to the build? I mean, twenty-twenty, dude, it's not that long ago. A billion in revenue, and you started in twenty-twenty.

A Yes, so the way it worked was, so I've always been a really big fan of MVPs, and so I literally just built myself or be one in a couple weeks. Like, I think the really nice thing was, so again, I had worked in this space for a really long time, so I already had a very clear vision of what I wanted to build. So as opposed to feeling like I needed to go out and hire 10 engineers in order to build a product, instead of feeling like I needed to go out and fundraise, you know, 10, 20, thirty million dollars in order to hire, you know, more people. Uh, again, I just wanted to build it myself, and I wanted to talk to customers myself, and so that's what I did. And so I think I, I, I've already built the V one in a couple of weeks. I, I posted about it on my blog. I told people about it that I, that, that I met and yeah, there actually was this giant demand for the data already. Um, so I think we were, we were very lucky early on.

AI assessment note: “I literally just built myself or be one in a couple weeks”

Q Do you think AI turns 10 X engineers into hundred X engineers, or average one X engineers into 10 X engineers?

A Or maybe both today, but definitely even more so in the future. I tend to think of It's like good people have so many ideas that they just don't have time to implement. And if you think of AI today as something that isn't necessarily coming up with the greatest ideas, although it can, but it often just removes a lot of the drudgery of like your day-to-day work, a lot of your day-to-day coding. And so if you don't have to spend that time on the drudgery, but you just have like these endless ideas that are just bouncing around your head and AI just helps you put them to paper. Then I do think it kind of disproportionately favors people who are already like the the Tenex engineers.

AI assessment note: “Or maybe both today, but definitely even more so in the future.”

Q huge demand for your product, this is even more so the time when everyone goes, now raise money. Hire CS teams, hire sales teams, hire big, why did you not raise money then? I get it at the start when, hey, you didn't want to do what everyone else did. Why not raise money when it was a hair on fire problem and you had so many people calling you?

A I mean, I would say there was nothing that raising would help us with. Again, we were, we were very lucky to be profitable from month one. And so we didn't need the money. We didn't need a sales team. Like I didn't, I actually didn't want a sales team going out and selling our product. Like I wanted people to buy us precisely because they understood the value of high quality data. They saw all the gains that our data was producing. I didn't want to buy it. I didn't want them to buy us simply because they heard about us in some tech cartonical, because that would almost put them at odds with the kind of product that we were building. Like one of the things that I think is actually really important is that you want customers who Especially early on. You want customers who believe in your product and not people who are simply giving you a little bit of money. Like, because your early customers will shape the kind of product that you're building, Cause yeah, you're building for them. You're, you're building for their needs. Like they're giving a lot of really, really great feedback. And so you almost want customers who share the shame overall vision. And so that was actually very important for us. Like I, I didn't want sales teams who would email 10,000 people and be like, Hey, any, any thoughts on, on getting good data? And it was just very, very counter to the kind of, kind of pr…

AI assessment note: “there was nothing that raising would help us with. Again, we were, we were very lucky to be profitable”

Q Do you worry that by not being so ingrained in social, you miss out on certain elements that is important to be in? Or do you think that purity of mind that you get is really so valuable?

A It's kind of funny because, again, I used to work at Twitter, and I, I love Twitter back in the heyday. But I actually really am glad that I'm not surrounded by default ways of Silicon Valley thinking. So every now and then, if something is important enough, like maybe there's some big new product is actually really cool, or there's some really, really interesting new research paper. It'll be like big enough that even though I'm not monitoring Twitter every day, it will just reach me in some other way. Like, yeah, one of our employees will tweet or like post it in our stock channel, or somebody will email it to me. So like the really important stuff will manage to percolate itself to me. Um, in other ways, but I actually am really glad that I'm not worrying about what people are saying about us on Twitter.

AI assessment note: “the really important stuff will manage to percolate itself to me”

Q Can I ask you, Edwin, you are very composed as a leader, as a CEO, you know, really it translates incredibly. Where are you not meeting the bar? Where are you not great and you are aware of it?

A So I think one area where I'm not great, which is kind of funny, but, uh, one area where I'm not great is I'm really bad at understanding financials. So sometimes people around the company, they'll try to tell me, Like, hey, do you, have you been paying attention to our revenue numbers? Have you been paying attention to our costs? Have you been paying attention to our margins? Do you even know what they are? And I don't. Like, uh, they're just like these financial matters that, like, I, I, I could not tell you what EBITDA is. I mean, I know, I know the acronym stands for, but the difference between that and revenue and profit and Net margin and, like, I actually just don't know any of these terms, and it's just like this blind spot. Like, no matter how much I tried to try to understand these things, I just can never remember. Um, and so...

AI assessment note: “one area where I'm not great is I'm really bad at understanding financials”

Q Do you think you have a fundamentally different business then? Because you're all lumped in the same category. But if they're passing along a warm body, and you're passing along data, it's a phenomenally different product, and it's monetized differently, no?

A Yep, yep. Yeah, again, like, if I think about the way we think about it, it's, it's maybe the following. Like, so we have always started out with quality of the data as our number one principle. And as a result, we need to build a lot of technology in order to, to measure that and improve that. If I think about like what, what goes wrong, it's that people often just don't realize how difficult quality control is. And people often think that humans are smart. And so if you just throw a bunch of humans out of the problem, you'll get good data. And what we found is that's, is that, that, that, that is completely untrue. Like for example, I went to MIT, but yeah, I think half of the people who graduate with a CS degree, they, they can't even code. So it's a really challenging problem to detect high quality. And second, if you actually take the folks from MIT who can code, they're actually just going to try to cheat you. They're going to sell their accounts to somebody in a third world country. They're going to try to use LLMs to generate the data for you. They're going to come up with all these crazy methods to cheat a system. So it's also this really, really challenging problem to detect low quality. It's actually really adversarial. And so what we found is that when you want to get the highest quality data to train LLMs that are already, you know, super intelligent, you actually …

AI assessment note: “we have always started out with quality of the data as our number one principle.”

Q What does that mean? I'm sorry, if you just play that out, what does that landscape look like then? Because OpenAI and Anthropic are so unique in their properties and characteristics. It means there'll be 10 more of them. What does that look like?

A I don't know if there'll be 10 more of them, but I can certainly see even like three more of them, and I just think each one will have Different trade-offs that they're willing to make, different focuses that they'll have, and, like, even today, like, Claude is really, really good at coding. Claude is really, really good, I think, at enterprise and, like, instruction following, whereas ChatGPT is, yeah, it's, like, more optimized for consumer use cases. Like, I think it actually has a really, really great and fun personality right now. And then Grok, like, yeah, Grok is willing to maybe answer certain questions that maybe, maybe it should, maybe it shouldn't, but it's willing to, uh, be a little bit transgressive in ways I actually think are very, very, Interesting. And so I actually think that, um, just like this willingness to have different personalities and different boundaries and different focuses on your models that, that just leads, um, that leads to models to be good at different use cases, just in the same way that like, yeah, there's like, I think analogy is there isn't a single poet. There isn't a single mathematician that is like the, the greatest mathematician of all time. They all have different focuses. They all have different ways of approaching these problems. And I, I think that richness, uh, like what we often call like richness of human intelligence, that w…

AI assessment note: “I just think each one will have Different trade-offs that they're willing to make”

Q What do you think no one knows about working within these big, incredibly hailed companies that they should know?

A I think one of the things that people don't realize, again, from the outside, it's that how much of what you're building, again, is for this internal company machinery. And how much of the internal company machinery is simply because a lot of people within these organizations, their goal, again, their goal isn't to build a product. Their goal is to tell their friends they're, they're, they're, they're a VP of a thousand person org. And that sounds impressive. And so their, their goal is to think about, okay, so how do I, how do I grow my org even faster? How do I find more teams that I can hire? How do I, Have these monthly performance reviews where, again, now that I've built this thousand person org, I need to prove to my VP, my CEO, that the thousand person org I'm building is efficient and useful. And so, uh, like basically a lot of the work that goes on in these drone companies, these large companies, it is simply to kind of perpetuate and grow even, even further a lot of this, like very, very big company machinery that It exists purely, purely for, like, internal reasons.

AI assessment note: “how much of what you're building, again, is for this internal company machinery.”

Q Can I ask you, in terms of meeting cadence, I'm sorry for being granular and I told you we go off schedule, but I've had Toby on the show in the past who, from Shopify, who's obviously advocated for no meetings. Given the ability to spend lifetimes in meetings that are quite pointless, how do you approach meeting policy and what does and doesn't belong in the org?

A Yeah, so I, I'm a big fan of that. Um, so like I, for example, personally, I actually have no one-on-one meetings and it's kind of funny because oftentimes people will ask me, well, How often do you meet with your reports? How often do you set aside for, like, for these meetings? I just don't have them at all. Like, oftentimes, like, I will just give people my calendar, my Calendly, and they're just surprised at how blank it is because I try to avoid filling my meetings all day. And so I will actually go out and I, like, I, like, sometimes when people join, they'll be like, okay, um, I need to go and have one-on-one meetings with these 10 other people that I'm going to collaborating with on a weekly basis. That's just because that's so used to when you come from Google or Facebook. And I tell them, why are you having these standing one-on-one weekly meetings? Like, did you not talk to them every day during Slack? Did you, like, are you just like unaware of what they're doing? Like, it's almost like a negative sign if you're having a one-on-one weekly meeting, because it means that you just don't know what's going on with these people. You're not, you're like almost waiting for your weekly meeting to raise, raise interesting questions and raise interesting problems. And so I think we're, we're pretty ruthless internally about killing meetings when they're unnecessary.

AI assessment note: “we're pretty ruthless internally about killing meetings when they're unnecessary.”

Q Do you think AI turns 10 X engineers into hundred X engineers, or average one X engineers into 10 X engineers?

A Or maybe both today, but definitely even more so in the future. I tend to think of It's like good people have so many ideas that they just don't have time to implement. And if you think of AI today as something that isn't necessarily coming up with the greatest ideas, although it can, but it often just removes a lot of the drudgery of like your day-to-day work, a lot of your day-to-day coding. And so if you don't have to spend that time on the drudgery, but you just have like these endless ideas that are just bouncing around your head and AI just helps you put them to paper. Then I do think it kind of disproportionately favors people who are already like the the Tenex engineers.

AI assessment note: “disproportionately favors people who are already like the the Tenex engineers”

Q Okay, so you realize this data problem in twenty-twenty, you leave Twitter. What happens then? You go heads down into product build for several months. You go about recruiting the first team members. Can you just take me to the build? I mean, twenty-twenty, dude, it's not that long ago. A billion in revenue, and you started in twenty-twenty.

A Yes, so the way it worked was, so I've always been a really big fan of MVPs, and so I literally just built myself or be one in a couple weeks. Like, I think the really nice thing was, so again, I had worked in this space for a really long time, so I already had a very clear vision of what I wanted to build. So as opposed to feeling like I needed to go out and hire 10 engineers in order to build a product, instead of feeling like I needed to go out and fundraise, you know, 10, 20, thirty million dollars in order to hire, you know, more people. Uh, again, I just wanted to build it myself, and I wanted to talk to customers myself, and so that's what I did. And so I think I, I, I've already built the V one in a couple of weeks. I, I posted about it on my blog. I told people about it that I, that, that I met and yeah, there actually was this giant demand for the data already. Um, so I think we were, we were very lucky early on.

AI assessment note: “I literally just built myself or be one in a couple weeks.”

Q Can I ask, going back to that story then, so you build out the MVP, you post it, and then you said, luckily, you said it very nonchalantly, Edwin, which is very sweet, like, people came and people liked it. What did that look like? How did the initial demand come to you?

A Sorry. I think I say it nonchalantly because it felt very nonchalant. Um, I think what would end up happening is, so I would, uh, find all these people who really were desperate for a lot of really high quality data. And I mean, the way it work is they would just email me with their request, or we would just jump on a, jump on a live meeting and we would just get started. And it might take a week or a couple of weeks to negotiate some sort of, um, SOW or contract just because, you know, a lot of this does have to live within the confines of their company. But, um, yeah, I mean, I think we're really lucky in that I had, again, I had a lot of experience in this space. And so I had a lot of experience working with ML engineers and research scientists and the ways that they wanted to get data and the way they wanted to look at it. And so, um, I think, I think things just moved very, very, very, very quickly.

AI assessment note: “they would just email me with their request, or we would just jump on”

Q huge demand for your product, this is even more so the time when everyone goes, now raise money. Hire CS teams, hire sales teams, hire big, why did you not raise money then? I get it at the start when, hey, you didn't want to do what everyone else did. Why not raise money when it was a hair on fire problem and you had so many people calling you?

A I mean, I would say there was nothing that raising would help us with. Again, we were, we were very lucky to be profitable from month one. And so we didn't need the money. We didn't need a sales team. Like I didn't, I actually didn't want a sales team going out and selling our product. Like I wanted people to buy us precisely because they understood the value of high quality data. They saw all the gains that our data was producing. I didn't want to buy it. I didn't want them to buy us simply because they heard about us in some tech cartonical, because that would almost put them at odds with the kind of product that we were building. Like one of the things that I think is actually really important is that you want customers who Especially early on. You want customers who believe in your product and not people who are simply giving you a little bit of money. Like, because your early customers will shape the kind of product that you're building, Cause yeah, you're building for them. You're, you're building for their needs. Like they're giving a lot of really, really great feedback. And so you almost want customers who share the shame overall vision. And so that was actually very important for us. Like I, I didn't want sales teams who would email 10,000 people and be like, Hey, any, any thoughts on, on getting good data? And it was just very, very counter to the kind of, kind of pr…

AI assessment note: “there was nothing that raising would help us with. Again, we were... profitable from month one.”

Q a dude. I interview, I've interviewed a thousand founders in the nicest way. I've, I've almost never met a founder like you. And so I, in, in a nice way, it's really special, but with a pure mindset, like you have, what are you doing it for then to build a business that you can pass on to the next generations to build a legacy? What is it for you?

A I mean, I, I think it really is to help achieve AGI. Like, if you think about every, every, like, what do, what do kids dream of? Like, yeah, when you're a kid, you literally dream of building AI that can do all these amazing things, and now we have the chance to do it. Like, I really do think we are such a critical aspect of what all these companies are building. Like, again, a lot of our customers at these Frontier Labs, they will just often tell me they wouldn't be able to build what they're building without us, and they're just amazed at what we do. And so, Being able to be this critical part of what is Literally the greatest technology of both our time now, but also maybe one of the most important things we can ever build. That's, that's amazing. And so why would you, why would you get acquired and stop doing that? Because yeah, getting acquired would be really limiting. It would be this admission of failure and jumping ship because you can't make it on your own anymore. When we're the opposite, we're incredibly successful and there's literally nothing else that I'd want to do instead.

AI assessment note: “I think it really is to help achieve AGI.”

Q If compute continues to prove to be the unlock, where throwing more compute at it unlocks more and more performance, does that denigrate data quality in the prioritization stack?

A I mean, I actually just fundamentally don't believe that you can throw more compute out of the problem, because if you're not getting the data that the computer is essentially trained on, or if you don't have the right objectives and evaluation metrics that again, your, your computer is optimizing towards, you're just going to fall into this trap of seeing progress that actually isn't there. Like, I can give you some examples. So let me talk about why I think data quality is such a problem. So I think data quality issues have already been a huge setback for a lot of Frontier Labs. Like, one of the things that we often hear from teams over and over is that before they used us, they tried getting data in other ways, and so they'd train their models, they'd evaluate their models, and their metrics kept going up. But after six months or even a year, they realized that they're trading data with shit. They're evaluating data with shit. And so all the progress that they Thought they were seeing was actually completely misleading, and they either made no progress or their models after six months were even worse than when they started. Like, for example, we see this a lot with LM Arena. So LM Arena is this popular leaderboard of LM models, and it's basically the equivalent of clickbait. What happens is that you have people going on to what's called a chatbot arena. They'll enter a promp…

AI assessment note: “I actually just fundamentally don't believe that you can throw more compute out of the problem”

Q Do you worry that by not being so ingrained in social, you miss out on certain elements that is important to be in? Or do you think that purity of mind that you get is really so valuable?

A It's kind of funny because, again, I used to work at Twitter, and I, I love Twitter back in the heyday. But I actually really am glad that I'm not surrounded by default ways of Silicon Valley thinking. So every now and then, if something is important enough, like maybe there's some big new product is actually really cool, or there's some really, really interesting new research paper. It'll be like big enough that even though I'm not monitoring Twitter every day, it will just reach me in some other way. Like, yeah, one of our employees will tweet or like post it in our stock channel, or somebody will email it to me. So like the really important stuff will manage to percolate itself to me. Um, in other ways, but I actually am really glad that I'm not worrying about what people are saying about us on Twitter.

AI assessment note: “I actually really am glad that I'm not surrounded by default ways”

Q But before we, I do want to touch on kind of the working hours that you mentioned that, but everyone poses synthetic data as a big threat. And what happens to your business when we have synthetic data, That is obviously created, uh, automatically and labeled automatically. How do you think about the role of human labeled data in a world of predominantly synthetic data? What's your thoughts there?

A So I think synthetic data is actually really useful in some places, but I think people overestimate what it can do. So I'll, I'll give a couple examples. So right now there are a bunch of models that have been trained really heavily on synthetic data. But like I mentioned earlier, it means that they're only good at very academic, homework style, benchmark style problems. They're actually terrible at real world use cases. So yeah, synthetic data, it's made models good at synthetic problems, not, not real ones. And we actually hear from a lot of companies who tell us they spent the past year training their models on synthetic data, but if only now just realize all the problems that's caused. And so they've spent actually months throwing a lot of it out. Like a lot of them tell us that even a thousand or a couple of thousand pieces of really high quality human data that we generated for them, it's actually been worth more than ten million pieces of synthetic data. And so a lot of the work that we do, it's simply cleaning up all this synthetic data. And if you think about like why this happens, it essentially is because the models collapse on this very, very narrow scope. Of, um, of, like, similarity that the synthetic data creates, and so it just doesn't give the models the kind of the diversity and generalizability that they need. And then one other point is that there's also thi…

AI assessment note: “really high quality human data that we generated for them, it's actually been worth more”

Q What do you mean by, what do you mean by body shops and body shops masquerading as technology companies? I get it, but a lot of people criticize the space with this and say, oh, it's just labor camps, or it's, so what do you mean by body shops or body shops masquerading?

A A lot of companies in this space, so they don't have any technology. And when I think about technology, it's like they don't have any way of measuring quality of the data that they're producing, and they don't have any way of improving the quality of data that they're producing. They are literally just body shops in a sense that, like, they sometimes literally have no technology at all. They don't have a platform where workers are doing work, and so what they're doing is they're simply finding people, like, they're recruiting warm bodies, they're looking at resumes, like, anybody with a PhD, they'll just, like, instantly hire them. And then just passing them along to, um, to, to the, to the AI companies, to the frontier labs. And so again, they have no technology. They have no way of measuring what any of these workers are doing. They have no way of knowing if they're doing a good job or not. So they have no way of doing things like, Hey, what if I AB tested this algorithm for improving quality? What if I changed this method of allowing workers through? What if I tweaked your tools in order to change these questions around? Would it make your workers more efficient? Would it improve their quality or would it actually make it worse? They just had no way of doing these things because again, at the end of the day, what you're passing to the, to like their customers is just the bod…

AI assessment note: “they don't have any way of measuring quality of the data that they're producing”

Q What do you think no one knows about working within these big, incredibly hailed companies that they should know?

A I think one of the things that people don't realize, again, from the outside, it's that how much of what you're building, again, is for this internal company machinery. And how much of the internal company machinery is simply because a lot of people within these organizations, their goal, again, their goal isn't to build a product. Their goal is to tell their friends they're, they're, they're, they're a VP of a thousand person org. And that sounds impressive. And so their, their goal is to think about, okay, so how do I, how do I grow my org even faster? How do I find more teams that I can hire? How do I, Have these monthly performance reviews where, again, now that I've built this thousand person org, I need to prove to my VP, my CEO, that the thousand person org I'm building is efficient and useful. And so, uh, like basically a lot of the work that goes on in these drone companies, these large companies, it is simply to kind of perpetuate and grow even, even further a lot of this, like very, very big company machinery that It exists purely, purely for, like, internal reasons.

AI assessment note: “how much of what you're building, again, is for this internal company machinery”

Q Can I ask you, in terms of meeting cadence, I'm sorry for being granular and I told you we go off schedule, but I've had Toby on the show in the past who, from Shopify, who's obviously advocated for no meetings. Given the ability to spend lifetimes in meetings that are quite pointless, how do you approach meeting policy and what does and doesn't belong in the org?

A Yeah, so I, I'm a big fan of that. Um, so like I, for example, personally, I actually have no one-on-one meetings and it's kind of funny because oftentimes people will ask me, well, How often do you meet with your reports? How often do you set aside for, like, for these meetings? I just don't have them at all. Like, oftentimes, like, I will just give people my calendar, my Calendly, and they're just surprised at how blank it is because I try to avoid filling my meetings all day. And so I will actually go out and I, like, I, like, sometimes when people join, they'll be like, okay, um, I need to go and have one-on-one meetings with these 10 other people that I'm going to collaborating with on a weekly basis. That's just because that's so used to when you come from Google or Facebook. And I tell them, why are you having these standing one-on-one weekly meetings? Like, did you not talk to them every day during Slack? Did you, like, are you just like unaware of what they're doing? Like, it's almost like a negative sign if you're having a one-on-one weekly meeting, because it means that you just don't know what's going on with these people. You're not, you're like almost waiting for your weekly meeting to raise, raise interesting questions and raise interesting problems. And so I think we're, we're pretty ruthless internally about killing meetings when they're unnecessary.

AI assessment note: “we're pretty ruthless internally about killing meetings when they're unnecessary”

Q Most founders have a challenge where they need to hire now, but they haven't found the perfect person. And so they hire a seven out of 10. They let the quality bar slip because they need someone in the role. How do you think about that? And what would you advise them?

A Yeah. I think the funny thing is like, again, I've been at all of these other companies. Oftentimes when people are saying like, yeah, I, my hair is on fire and I really need this engineer. So I know they don't meet the bar. I'm going to lower the bar to hire them. Like actually the, the engineer didn't like, what are they doing? They're building probably a feature that nobody cares about. They're building an internal tool to improve the productivity of everybody around the company by two percent. While at the same time having so many meetings with them that they like take up five percent or 10% of the time just talking about the feature. Like a lot of the things that people hire for just actually aren't all that important. And so again, like when you don't feel like you have to hire for the sake of hiring, like when you have the mentality that Okay, if your company only grows by 10% or even zero percent, that's actually positive. Like, I think people right now, they have this view that if someone would tell you, oh yeah, my, my engineering org only grew by two percent this year, your initial reaction is going to be, okay, you guys must not be doing well. Right. And so there's like this negative incentive where people feel like they need to hire just in order to prove to other people that, that their business.

AI assessment note: “a lot of the things that people hire for just actually aren't all that important”

Q Is there a time when you let quality slip in any area of the company? And with hindsight, what did you learn from that?

A I, I think we've, we've never let quality slip. I mean, it's so, it's such a principle ingrained to, into everybody at the company. Like one of the things that we simply tell everybody when we first join, quality is the most important thing. It's more important than, uh, yeah, it's more important than anything else. If you have to make a deadline slip, because for whatever reason, you don't think the quality is there. If we have to say no to a project because We just can't handle it right now. I mean, uh, we can generally handle a lot of things, but for like, we, we just want to ingrain this principle that is okay to say no. It is okay to, um, kind of like let other things maybe slip just because we, we care about quality at the end of the day.

AI assessment note: “I think we've, we've never let quality slip.”

Q Can I ask, going back to that story then, so you build out the MVP, you post it, and then you said, luckily, you said it very nonchalantly, Edwin, which is very sweet, like, people came and people liked it. What did that look like? How did the initial demand come to you?

A Sorry. I think I say it nonchalantly because it felt very nonchalant. Um, I think what would end up happening is, so I would, uh, find all these people who really were desperate for a lot of really high quality data. And I mean, the way it work is they would just email me with their request, or we would just jump on a, jump on a live meeting and we would just get started. And it might take a week or a couple of weeks to negotiate some sort of, um, SOW or contract just because, you know, a lot of this does have to live within the confines of their company. But, um, yeah, I mean, I think we're really lucky in that I had, again, I had a lot of experience in this space. And so I had a lot of experience working with ML engineers and research scientists and the ways that they wanted to get data and the way they wanted to look at it. And so, um, I think, I think things just moved very, very, very, very quickly.

AI assessment note: “they would just email me with their request, or we would just jump on”

Q And so, in the early days, everyone else is acquiring supply side of talent, correct? All the other people that compete in the space. And you're not acquiring that talent supply, you're building product, correct?

A Um, I mean, it was both because, I mean, obviously we need a talent supply in order to make our product work, but again, it was less about, so, so there are some companies in this space who will simply think of it as a pure supply problem, and they don't give any consideration to the technology, like both the technology, the underlying technology, like, how do you identify these people? How do you make sure that they're doing good work? How do you remove the bad quality work? Like, they're just literally not They're not thinking about any of the technology aspects at all. And they're also not thinking about the product at all. Like how do you present the data to the customers? Like one of our principles, like one of the principles that I've always had, even prior to surge when I was just an ML engineer or data scientist, one of the things that I've always tried to encourage is what we call this visceral understanding of the data. Like I really just want you to go in and get your hands dirty and look at the data. Like historically, a lot of ML engineers, they kind of just don't take the time to look at the data. And maybe that's because the data just isn't all that interesting. Like, when all you're doing is drawing bounding boxes on cars, sure, I don't need to look at a thousand bounding boxes, but when you're doing is, yeah, creating poetry, creating mathematical equations, cr…

AI assessment note: “it was both because, I mean, obviously we need a talent supply in order”

Q So you think AI is much more dangerous than we let on?

A Dangerous, but that it can be accidentally maximized towards the wrong objectives that like today, okay, sure. Like if you maximize towards these LM arena objectives or benchmark hacking, you're like the worst that will happen is that your models were regress in progress a little bit, but like the more fundamental problem is that people don't realize this. And so in the future, when the models are more powerful, And yeah, you're, you're basically accidentally maximizing AI models towards the wrong objectives, and you just have no idea what will happen. Almost like a similar, similar phenomenon to what's happening today, but because the AI models are so much more powerful. Like, yeah, they're literally building the code for an insurance company, or they're literally building the code for, you know, some trillion dollar, uh, some trillion dollar company. Just the consequences can be much worse.

AI assessment note: “Dangerous, but that it can be accidentally maximized towards the wrong objectives”

Q What do you mean by, what do you mean by body shops and body shops masquerading as technology companies? I get it, but a lot of people criticize the space with this and say, oh, it's just labor camps, or it's, so what do you mean by body shops or body shops masquerading?

A A lot of companies in this space, so they don't have any technology. And when I think about technology, it's like they don't have any way of measuring quality of the data that they're producing, and they don't have any way of improving the quality of data that they're producing. They are literally just body shops in a sense that, like, they sometimes literally have no technology at all. They don't have a platform where workers are doing work, and so what they're doing is they're simply finding people, like, they're recruiting warm bodies, they're looking at resumes, like, anybody with a PhD, they'll just, like, instantly hire them. And then just passing them along to, um, to, to the, to the AI companies, to the frontier labs. And so again, they have no technology. They have no way of measuring what any of these workers are doing. They have no way of knowing if they're doing a good job or not. So they have no way of doing things like, Hey, what if I AB tested this algorithm for improving quality? What if I changed this method of allowing workers through? What if I tweaked your tools in order to change these questions around? Would it make your workers more efficient? Would it improve their quality or would it actually make it worse? They just had no way of doing these things because again, at the end of the day, what you're passing to the, to like their customers is just the bod…

AI assessment note: “they're simply finding people, like, they're recruiting warm bodies”

Q we start on the story itself, and pre, actually, the founding of Surge, you said to me that 90% of the people while you were working at your Google, your Facebook, your Twitter, 90% of the people there were working on useless problems. I thought that was a very interesting place to start. Why were they working on useless problems, and what did it teach you about efficiency seeing that?

A Yeah, so I think the biggest lesson for me was that you can build a completely different kind of company with 10% of the resources and 10% of the people, but you're still moving 10 times faster and building a 10 times better product. Like, imagine you could just magically move to 90% of people who aren't working on interesting problems. What would happen then? Well, if you have a company that's one-tenth this size, you don't need to hire as many people, so you spend less time interviewing, you spend less time in meetings, you spend less time giving people updates for the sake of updates. And if it's one-tenth the size, that means everybody has a better view of what's going on around the company, because there isn't all this clutter, masking the important stuff. And because the talent density is higher and the teams are smaller, that means the communication is a lot higher and the iteration speed is a lot higher and better ideas just percolate around more quickly.

AI assessment note: “you can build a completely different kind of company with 10% of the resources”

Q intensity Required to win in terms of work ethic. You must work seven days a week if you want to build a ten billion dollar plus company, and the ability to put your phone on the side and not check an email does not exist anymore if you want to build a ten billion dollar plus company. You've built a ten billion dollar plus company. Do you agree with me?

A So I would say I think you have to be willing to work hard. Like, you have to be willing to jump on a call at two a.m. And yeah, customer, like, I think one of the things that I love is that sometimes customers don't call me. They'll literally call me at two a.m., three a.m., and you're like, hey, our models are freaking out. I need a bunch of data to fix it by six a.m. Can you do it? And maybe going back to the question of things that make me happy, like nothing makes me happier than knowing that, yeah, we can, we can deliver those. Like, yeah, we can deliver 10,000 data points to you. In the next few hours, even if you call us at three a.m. to fix some critical, critical bug, critical fire that you're facing. That is actually something that makes me incredibly happy. And so I think you have to be willing to work hard. I think a lot of people do confuse working hard with creating value. Like, like again, I mean, it's maybe a trope to say, but you have to work smart and not just hard. Like, if I think about a lot of what I'm doing, oftentimes the best ideas come to me when I'm just walking around, not necessarily when I'm at my computer. And so, I think we, I mean, I think we all work really hard, but I, I, I wouldn't confuse the number of hours we spend with actual progress.

AI assessment note: “I wouldn't confuse the number of hours we spend with actual progress.”

Q I spoke to Garrett at Handshake right after the acquisition. He said, like, I'm just staying up all night. There's just a tidal wave of scale customers moving to us. Did you have the same though in terms of that tidal shift in customer demand shifting to you as well as the realization that you mentioned there?

A Yep. I mean, so I would say I'm pretty sure that a lot of these other companies, they are, um, Like at the end of the day, people want high quality data and they don't want to be working with body shops. And so. I think we've seen, like, a massive wave interest because, like, yeah, like, the space is really large, and there are a lot of teams who are still using scale for legacy reasons. It's like, at the end of the day, we were already the biggest and best in this space, and so even when there were teams at some of these larger companies who weren't working with us already, they, like, they kind of, like, knew who to turn to.

AI assessment note: “I think we've seen, like, a massive wave interest”