The Exchanges

Q My timelines are wrong. Um, are there, like, a few, um, specific learnings or breakthrough moments, you think, since beginning to release that are interesting?

A Yeah, I mean, there's been, it's, it's, it's been a slog over many years, or over multiple years at this point with, with, with, with many, many learnings. I would say, yeah, I mean, just to give you a timeline of the arc of what, what we shipped is, you know, so the first thing was our, our writing system, uh, we called it AI Writer. Um, that's the first thing we launched. Uh, it was easiest to get working because it's like single step task, rewriting, editing text, uh, there's no like retrieval aspect. It was just like raw access to the model to write, uh, Uh, to write the text. The next, the next big thing that we immediately started working on was a Q&A. Doing a semantic index of the entire workspace, and then letting you ask a question, and it can give you an answer that's, that's grounded in the sources. That was also immediately obvious to us that, that'd be super useful, and so we started work on that. That one we launched in, I think it was October, 23. So we started a beta before then, but then our, our GA was in October. That was a much bigger effort to get working, obviously. We weren't just like, like, plugging in the LLM. It was actually Doing this, like, real-time updating index.

AI assessment note: “just to give you a timeline of the arc of what, what we shipped”

Q What about on the Notion agent side? Like, do you have a workflow there that is core to daily work?

A Yeah, I mean, I mean, I, I use our personal agent all the time, so it's, It, it has all the context about, about our company and everything that's going on, you know, so like, uh, for example, last night I was asking you about, um, how the custom agent launch was, was going and like, like, like what the, what the signals we're getting from it, uh, we're super useful for that. And then for, I, I have many custom agents that are, that are running. Um, my, my, my personal favorite is I have a email triage agent. So it has access to all of my work and personal emails. Um, And it just, uh, wakes up every day and just archives all the stuff I don't need to see. I trained it over time to, uh, to, to learn my preferences.

AI assessment note: “my personal favorite is I have a email triage agent”

Q How did you decide that this was going to be an in-house effort versus a partner effort? That given most people who made cars said we're going to go partner or buy something here.

A I guess the emotional slash philosophical is on things that are really important, we've taken the approach of vertically integrating them. So electronics, our software, all the high voltage systems in the vehicle, so things like motors, inverters, Uh, all the power electronics. These are all things we, we develop and build in house. And in a few cases, you know, we had to start with something that was either off the shelf or partially off the shelf, but today, all of that's completely in house. And in the case of self-driving, we knew that long-term it needed to be something that was developed internally. We started, as I said, with a mobilize centric solution, which a lot of folks did. Right. Particularly in like, you know, that 2015 to 2021 timeframe. But when you really look at what's necessary to, to be successful, In a neural net based approach, there's a core set of ingredients that very few people have, and I think we uniquely have them. So first and foremost, you need to have complete control of the perception platforms. You have all the, everything that the, the, the system is capable of observing, whether that's cameras, radars, or lidars, or some combination of all three, you need to control that, meaning there's no intermediary company that's like processing some of their information. And so that's powerful because you can then Feed raw signals into your system. The…

AI assessment note: “on things that are really important, we've taken the approach of vertically integrating them.”

Q think, you know, coming full circle, that's a very interesting opportunity for, you know, MongoDB in particular, because the, like one of the reasons you might actually replace these systems of record in this age is you want to keep much richer information about interactions, or whatever else it is in your system of record, and it's messy, it's got to sit something, right? It might not be Oracle anymore.

A I was talking to a European retailer the day before yesterday at NRF in New York, and they said, they tried a bunch of systems of record for ERP, Expensive failed implementations, lots, lots of issues on supply chain all the way to financials, and decided they are going to invest in just building it themselves, and they are building that on MongoDB. And I mean, that's a great use case, and I'm like, okay, you had me at hello. And, ah, ah, to do that, but if these kind of organizations are going to transforms within or disrupt within, I asked, ah, Deepa, our CIO, the same question, that are there things that we can build ourselves to disrupt within on MongoDB? So that's the story and the compelling value we can articulate also to our customers.

AI assessment note: “decided they are going to invest in just building it themselves, and they are building that on MongoDB.”

Q Do you do that from a legal perspective, or is that something that's a little bit more in the future relative to where code is today?

A Yeah, we're starting to do this now, and actually, like, when I was at DeepMind, a lot of the RL research I did was that, and so when we first got access to GPT-IV, we had the very strong intuition of, okay, you're going to be able to, you know, string a bunch of these model calls or eventually do things like reasoning models where the full agent is differentiable, and Even the first day we got access to GPT-IV, Winston went in his room for 14 hours and just redid a bunch of his associate tasks, and when I looked at the work he was doing, it was essentially like this hacky agentic where he said, okay, I would need to go look up this case law, summarize it, take that summary, use it to draft, and so seeing him do that gave us the intuition very early on of that's the direction this is going, and you can kind of think of associates as Agents. They get this task from a partner that's, hey, I have this high level case strategy. I want to see if I can find a bunch of case law that supports it. Can you go research that? Look it up, cite it, write me a memo. And so a lot of the systems we're starting to build look a lot like that. And I think one interesting direction that the coding labs, the research labs are going is building these RL environments where you deploy these agents and they can interact with a code base and see if they can pass unit tests. And in legal, that RL environm…

AI assessment note: “Yeah, we're starting to do this now”

Q One of the, as you mentioned, like the labs are all very focused on, um, uh, RL scaling in, um, like coding and math domains. Um, I think those is like highly verifiable, right? Not perfectly so, but like how, how do you think about the appropriateness of like law for RL given it's not as easily verifiable?

A Yeah, this, this is one of the biggest problems. And I remember we had conversations early on when we were trying to figure out what is the right Evaluation structure. So I think the hardest thing about legal is most of these tasks are very long form text generation. And so there are definitely subsets of legal work that are super verifiable of go in this data room and just find all the change of control provisions that you can kind of build these traditional data sets. But for something like generate this merger agreement, it's really hard to just give some binary like this is good or this is bad. And I think this has been like a big research problem, like with all the labs we work with, and also internally, there is just this open question of how do you build that reward function? And if you think of what that reward function is at the law firms, it's the partners, right? Like at the end of the day, there is no way to verify this besides the senior partner who's done a bunch of these said, yeah, this looks pretty good. And so internally, these law firms have a bunch of data of here's all the edits that went into this and the feedback. And so We are starting to think about how do you use that to train these reward functions, but I would say that is one of the really big problems, but I think one of the interesting things is, I think you actually have the same problem in progra…

AI assessment note: “there is just this open question of how do you build that reward function”

Q So Benchling is a system of record company. It's a data platform. What is Benchling AI?

A Benchling AI has kind of two, two major components to it. The first is tools for simulation. So this is taking open source proprietary company's internal models and making them accessible to scientists directly in their workflow. So the right model at the right moment in the scientific workflow already set up so that a wet lab scientist without computational skills can use it effectively. And then the results are linked to all of their other information in Benchling. And then we also see that laddering up to being able to help scientists recommend, like help recommend for scientists the next best experiment to run based on all the work they've done in the past, plus all the public literature available. And so we think it's like an exciting way to approach the, the co-scientist problem. Then the other facet of Benchling AI is agents that automate work for you. And so we've released this deep research agent. It works similar to the deep research agents from, from Anthropic and other foundation labs. But what it does is it works over Benchling data with the context of the Benchling data model. And so it enables scientists to ask these very difficult and science is fundamentally about like asking and answering questions. And so for our customers, it helps them to do a type of question that previous in the past would have taken weeks or months to do and do that in just, you know, a …

AI assessment note: “Benchling AI has kind of two, two major components to it.”

Q What do you think just gets, just becomes part of coding tooling?

A I think there's going to be a class of tools where you sort of start from the front end. This would be things like Lovable or Bolt or Replit or maybe even Figma Make if you're coming from the design side, and you'll have like an all-in-one platform for Build an app or even like, honestly, like build a business, like put payments in it. It's like, it's kind of like the evolution of like either a Shopify storefront or like WordPress or Squarespace or something like that. So I think that's all going to be bundled more on like the core pro developer side. I can't tell if it's going to be a world of like MCPs and integrations and like all these tools sort of interplay. That's one approach, or it's going to be more like there's enough alpha and like You put all of these things together. And I think warp is a little bit more like this. Like we're trying to build a single pane of glass, for instance, for doing like local agents and remote agents.

AI assessment note: “I think there's going to be a class of tools where you sort of start”

Q And you just closed literally a billion dollars in new financing, which I think brings your total over time to 1.3 billion. What is that money going to go to? What products do you provide? What services? Like what, what are you currently doing?

A Yeah. So the capital really is going to accelerate our vertical integration, which allows us to ultimately lower costs for consumers. So our strategy at the highest level is to develop a compounding cost advantage through vertical integration. So we design batteries, we make them like literally manufacture them, we install them, own them, operate them, and we sell power directly to a homeowner. So to answer your second question specifically, when you sign up with base, we become your electricity provider. So today we're only available in Texas where now we, we sell power to Deregulated customers who can choose their electricity provider, and then we do sell our technology to the regulated utilities in Texas, and then they offer our services to their customers. So when you sign up with base, we install our battery on your home. When the grid's up and running, we use that battery to serve the grid. When the grid goes down, you get that battery to back up your home, and we're able to save our customers on the order of 10 to 20% a month on their electricity. So as we invest further, as we build new generations of the technology, our costs will go down, our returns will go up, we'll be able to Share those returns, so to speak, with the customer in the form of lower and lower prices, driving the price of, of the Electron down, driving the availability or the reliability of the Electr…

AI assessment note: “So the capital really is going to accelerate our vertical integration”

Q That sounds like an amazing mix. Are there specific areas that you're hiring for right now or looking for key talent in?

A Yeah. So as part of our announcement, kind of the big theme here is join the charge, right? It's a call to action to the most talented engineers, operators, and creatives in the world to come join us on what we think is the most interesting, exciting, and important mission out there in technology right now. And so we're really hiring across all teams, software, hardware, finance, go to market, business development. Regulatory policy, uh, deployments engineering, manufacturing engineering, you know, specifically some areas where we're really focused right now are firmware, power electronics, uh, mechanical engineering, you know, design engineering, uh, so all kinds of software engineering, basically everything from, you know, very low level firmware to, you know, front end to back end and cloud engineering and everything in between. So, um, you know, right, right now, as you, as you noted, you know, we've, we've raised, uh, you know, 1.3 billion in the last 18 or so months. We've got more demand than we can serve. We're standing up our first factory in Austin to produce our product at, at really large scale. And really the constraint is great talent. Um, and so, you know, we want to put the word out there that we're open for business and we, we want to bring, uh, more great people down to Austin to join our team.

AI assessment note: “specifically some areas where we're really focused right now are firmware, power electronics”

Q Um, A, do you think those projections are correct? And B, how do you think about how we solve for that?

A I do think directionally the demand projections are correct, and I, and I think that, or, or actually, uh, potentially understated. Uh, I think there's a massive amount of electricity demanded coming. Uh, now how do we serve it is the question. You'll hear people talk all the time about, oh, we're running out of energy. We have no energy. That's not really true. We're not using the energy that we have in the most effective way possible. Uh, and you know, another way to think about it is, you know, the U S grid kind of max peak is somewhere on the order of 700 gigawatts, but average Average demand across the country is closer to like, 300 gigawatts, right? So we've got another three, 400 gigawatts, uh, you know, just up to 700 gigawatts if you believe that, you know, that peak is going to grow to a terawatt of latent capacity. And, you know, the way to access that capacity is by time shifting electricity, right? Using batteries and software. And so, you know, we think that our technology layer is going to unlock a bunch of latent capacity on the grid, help us meet that demand, but we will also have to build more generation, right? And there's tons of Smart people and interesting companies that are working on.

AI assessment note: “I do think directionally the demand projections are correct”

Q and, um, one of the things that really stood out to me is the energy that you feel is, no pun intended, as you sort of walk around, everybody seems very motivated, very driven, very on it, and it's kind of like a buzzy space and culture. How do you guys, was that purposeful? Did that just happen through the people that you hire? Like how did you approach that?

A A bit of both. I think culture largely is, is the people that you hire and those people define kind of the early culture. And then you can, it's a bit of nature and nurture, right? You can kind of tend to the culture and, and, and make sure it, it gets better and moves in the direction you want it. And so I'd say, you know, the things that we really value as part of our culture, you know, the first thing that comes to mind is working with urgency and focus, right? And I think it's really easy to have It's like, yeah, we do nine nine six. We probably do more than nine nine six, but we don't really talk about it in that way. And it's like, just kind of natural to how we operate, but we have an extreme focus and we ruthlessly prioritize the most important things. Everyone at the company knows what the North stars of business are. They know how the thing they're working on ladders up to the North stars. We talk about our business very openly. So, you know, as you've seen, team has lunch and dinner together. Lunch, dinner, table conversations are really about like our long-term vision. Where are we going? What kind of new things should we be thinking about? Um, and you know, you walk around the office and the place looks like a Best Buy because there's just TVs everywhere as you've seen, right? Metrics everywhere. All the stuff that matters at the company is visible for everyone to …

AI assessment note: “A bit of both. I think culture largely is, is the people that you hire”

Q not actually rare, but they're called rare earth, and then they're fundamentally mined in a small subset of countries that actually have access to them. Should the US be changing its mining policy around this? Should we be, should Canada? I'm sort of curious how you think about how to address that, because there's a few different ways to approach it. One is just to mine more in certain places.

A China only emerged as a rare earth mining superpower about 10 years ago. It came out, uh, in twenty-fifteen with its Made in China twenty-twenty-five plan, and from that date onwards, we actually saw China's, um, refining activity of rare earth, uh, materials skyrocket. So what happened was they pursued a very aggressive industrial policy to build, uh, refinering capacity in China. Uh, and they started to flood the market, which sunk the price and started to really squeeze refineries located in the West, in, in, uh, Australia and Canada and in the United States. We actually have refineries. Historically, we have had refineries. And, uh, there is a huge refinery in Tennessee. Um, there, there are refineries in Arizona and Georgia. And, um, and so the, The solution to help put the genie back in the bottle is, I think actually the DoD's deal with MP Materials offers a good template, which is you need an anchor buyer and an end customer, and you need a price floor. So you need to agree with the supplier, in this case MP Materials, on a floor for a price, because what happens when these big off-take agreements, um, When, when a western refinery tries to compete with China, is China will artificially sink the price, the global price, depress it, in order to put western alternatives out of business, and then they raise the price again, which is classic monopolistic behavior. Um, we ca…

AI assessment note: “The solution to help put the genie back in the bottle is... a price floor”

Q Maybe six months ago, I think you, you, you said like, I think it's possible we have my definition of ASI in a couple of years. Um, do you still believe that's true?

A I, I still do believe that's true. Um, I think that where I think we'll be in a couple of years from now is that there'll be kind of definitive, um, super intelligence in, Some meaningful categories of work. And so, for example, when I say coding, I don't mean all of coding there, but there will be a super intelligence within some kind of slivers, some meaningful slivers of coding that are driving, um, I'd say immense progress in the companies that can benefit from that. And the reason why I would say that the problem of ASI would have been solved by then is because you've kind of, um, at that point, it's just a matter of operationalizing. Like what, you know, you know, it just so happened that these particular categories, like you might have a super intelligent front-end developer because there's so much data distribution for that on the internet, and it's easier to make synthetic data for that. But at that point, you have the recipe, and it's just a matter of, um, making kind of economic decisions of, is it worth sinking in X amount of dollars to get the data in this category, um, to get kind of something, um, close to super intelligence there. Um, an example of that is, um, What happened with reinforcement learning before language models? Um, effectively, the blueprint for building super intelligent systems was developed. It happened with, um, the Atari games, AlphaGo, um, y…

AI assessment note: “I, I still do believe that's true.”

Q Rippling is one of my favorite examples of, like, software companies that I think are very durable to, um, everything that's happening with AI. Does it change anything about your strategy, fundamentally?

A It definitely, um, makes us a lot more focused on data, um, like a lot of other companies, obviously, and, and capabilities around data. What I think is going to be very durable with, with AI is, like, everything that I've seen is, like, building the, the applications is, like, uh, I mean, I don't want to say it's, like, trivial, obviously everything's hard, but what's much harder than building AI applications are building, You know, governance and permissions and data pipelines and things like that. Um, and so, you know, that those are the areas like, you know, that we spend a lot of time on because they, they tend, they tend to be like our core strengths, particularly because like permissions and governance are very tied to an understanding of the work, you know, understanding, you know, usually permissions are about job and role and function. And or chart relationships. And so if you understand that deeply, you can be opinionated about who should have access to what, you know, who can do what in the system. And ultimately, I think like all of these AI agents are going to need to inherit the permissions of a person. Like some people would say, well, you know, maybe it can be like a service account that has distinct permissions. And the problem then is like, well, ultimately, like people are going to be interfacing with it. And if it has different permissions than they do, The…

AI assessment note: “It definitely, um, makes us a lot more focused on data”

Q of dev containers with some of your scientist teammates at the very beginning of the company. And both of you from the beginning, you know, talked a lot about platform investment. And so I actually think that's like a little bit sort of unconventional in terms of such a research oriented team to say, like, we need to make this platform investment. Can you talk a little bit about that?

A Yeah. Uh, so I, I've gone through the experience of going from, you know, zero to 100 on, on engineering, large engineering products before I, I worked on, uh, Stripe Link, which was kind of a multi-year project. And again, Stripe Capital where, uh, engineering teams scaling from zero to 25, 50 people by, by, by the time we were, were, were, were done there. Um, same, same for Link, maybe, maybe more. And. I think you just learn that unless somebody is really taking care to keep the entire system in their head and is an effective technical steward of the architecture, that things just evolve and the sort of the entropy of the software takes over and slows down your rate of progress to zero because nobody can, can get, get work done anymore. And so somebody needs to keep the entire system in, in their heads and the interaction between all those components and make sure that. People who are working on individual subcomponents of your code base actually have to minimize the amount of context they need to load into their heads to understand how to accomplish that task. So these are just the, the principles of really, you know, it's pretty basic. It's just simplicity and modularity, but making sure that's a practice and a kind of cultural, uh, cultural practice, and that everybody's on the same, same page about investing in that, uh, and that You know, people aren't cutting corners.…

AI assessment note: “I think you just learn that unless somebody is really taking care”

Q And I think it's not only the self-driving or the autonomy side that You, you folks really focus on, but I think the OS is really powerful. You sell, to your point, defense, construction, but a lot of your business is the giant automotive companies around the world. Could you talk a little about what you're providing through that OS and why it's beneficial and what does it actually do?

A When we refer to the vehicle operating system, we're talking about the full software stack that runs on these embedded systems that run on these vehicles. So, um, at the lowest level, right, we even do bootloaders because if you want to do very reliable updates to embedded systems, you have to control the bootloader. And then you go above that and we can talk about the actual, um, the technical gory details of the operating system itself. So think about like a true operating system. But on top of that, you have to have the middleware, which is responsible for, um, some abstraction, but also some really important safety critical data transport, uh, aspects of it. On top of that, you actually have the applications that are running and, uh, also many, many layers to those of course, but, uh, they end up really controlling the vehicle, um, but then also displaying information and information could be displayed To somebody who's in the vehicle, or it could actually be displayed to someone who's outside of a vehicle, let's say in the mining example.

AI assessment note: “we're talking about the full software stack that runs on these embedded systems”

Q The naming schemes in AI are, are something else. So you, you folks have a, a simplified version in some sense. Do you want to mention any of the highlights from four that you think are especially interesting or, you know, those things around coding and other areas? We'd just love to hear your perspective on that.

A By the benchmarks, four is just dramatically better than any other models that we've had. Even four Sonnet is dramatically better than three seven Sonnet, which was our prior best model. Some of the things that are dramatically better are, for example, in coding, it is able to Uh, not do it, uh, sort of off target mutations or over eagerness or reward hacking. Those are two things that people were really unhappy with in, in the last model where they were like, wow, it's so good at coding, but it also makes all these changes that I definitely didn't ask for. It's like, do you want fries and a milkshake with that change? And you're like, no, just do the thing I asked for. And then you have to spend a bunch of time cleaning up after it. The new models, they just do the thing. And, uh, and, and so that's really useful for professional software engineering where you need it to be maintainable and reliable.

AI assessment note: “Some of the things that are dramatically better are, for example, in coding”

Q and managing builds and scale of manufacturing, you're going to have many fewer form factors. And the other argument is the economic value of specialization is very high. And therefore there'll be, you know, thousands and thousands of different form factors as we move to sort of a robot driven future. Do you have a point of view on sort of where we're likely to land between those two viewpoints?

A I think we're going to gradient descending to optimization of productivity and efficiency. My hypothesis is that the requirements of different tasks are so vast that having very few form or, or sticking with one form is. Energy, energy inefficient. And a lot of tasks can be done and should be done by much more energy efficient form factors. Just an extreme and, and trivial example. If we put robots underwater, they should not be in the shape of humans. They better be in the shape of fish, right? Just think about energy efficiency. And the same with flying. I don't think human form is, uh, our airplanes are becoming more and more robots. Um, so I, I, I do think there's gonna be diversity.

AI assessment note: “I do think there's gonna be diversity.”

Q to date? And obviously there's still a lot of career to come, but I'm just sort of curious. I mean, obviously there's a lot of things that you did in terms of Sort of, uh, image and visual recognition-related systems and all sorts, but I'm just sort of curious, like, when you think, think of the last 20 years, what stands out the most, just given everything that you've done?

A Oh, thank you for asking that question. Of course, ImageNet is one of those, uh, ImageNet consists of multiple moments from the early struggles and being told I will not get tenure to, um, To actually realizing Amazon Mechanical Turk comes to rescue to the moment of Alex Nett winning, and also to a couple of years ago, I was at an event in Toronto with Jeff Hinton, and he said publicly, like, how that was so defining, and he, he was almost a little bit, um, apologetic, that image that was not As recognized as, uh, as neural networks. So that journey is very validating. And for scientists, the validation is not about recognition or awards. It's that you made a difference. Like that conjecture that no one believed in, that hypothesis that no one believed in, we were able to make it happen. So that's one thread.

AI assessment note: “Of course, ImageNet is one of those, uh, ImageNet consists of multiple moments”

Q to date? And obviously there's still a lot of career to come, but I'm just sort of curious. I mean, obviously there's a lot of things that you did in terms of Sort of, uh, image and visual recognition-related systems and all sorts, but I'm just sort of curious, like, when you think, think of the last 20 years, what stands out the most, just given everything that you've done?

A Oh, thank you for asking that question. Of course, ImageNet is one of those, uh, ImageNet consists of multiple moments from the early struggles and being told I will not get tenure to, um, To actually realizing Amazon Mechanical Turk comes to rescue to the moment of Alex Nett winning, and also to a couple of years ago, I was at an event in Toronto with Jeff Hinton, and he said publicly, like, how that was so defining, and he, he was almost a little bit, um, apologetic, that image that was not As recognized as, uh, as neural networks. So that journey is very validating. And for scientists, the validation is not about recognition or awards. It's that you made a difference. Like that conjecture that no one believed in, that hypothesis that no one believed in, we were able to make it happen. So that's one thread.

AI assessment note: “Of course, ImageNet is one of those, uh, ImageNet consists of multiple moments”

Q You've decided that you need to make it easier for normal humans who do not have, uh, infinite motivation to learn languages from computers. What was the first thing that you did that worked that like helped people get over the motivation hurdle of it being so boring?

A The first thing that we did that worked was making lessons, not 30 minutes long, but two minutes long. It makes a big difference. And it's not because people were spending less time. It's that Two minutes is amazing because at any point in time, you just think to yourself, oh, it's just two minutes. But the time investment for 30 minutes, you know, if you're right now and you're like, if I ask you, do you want to do something that takes 30 minutes? You're like, oh man, I don't know about that. Even though when you think the time investment is only going to be two minutes, you may spend the 30 minutes, but it's just, it's just like, yeah, I can start right now because I can end any time. So that's the first thing that worked. We did many other things that worked big, but that was the first thing that really was a big Big game changer.

AI assessment note: “The first thing that we did that worked was making lessons, not 30 minutes”

Q Chess now. Yes. How do you decide what Duolingo can teach beyond languages?

A What we're looking at for subjects to teach are, one, we're looking at very large audience. So we need something that hundreds of millions of people want to learn. There's a reason for that, and it's because we're an app at the end of the day, and we cannot charge 30,000 dollars To each user. We charge you 10 bucks. In order to make a significant amount of money from a new subject, it has to be learned by a lot of people. Otherwise, these very niche subjects just, you know, if we charge you 10 bucks and there's only a hundred people learning it, this is just not worth our time. Um, so they have to be, they have to be a large potential audience. Uh, the other thing is we look for things that take a long time to learn. Uh, you know, if you can learn something in two hours, you probably should just go watch a YouTube video and that does a perfectly good job. If it's just like two hours, we look for things that really take hundreds of hours to learn. Um, and then things that we think are, uh, are good for the world and we can do a good job with a mobile app. Oh, there's one extra thing, which is internally, we need to have somebody that's really excited about it. Um, so this has been true for all of these other subjects, you know, there's Somebody is very excited about music. Somebody is very excited about chess. And that, that's what has done it.

AI assessment note: “What we're looking at for subjects to teach are, one, we're looking at”

Q Did you have a, a favorite, like, most important task?

A We had a few tasks. People would just propose different tasks. One of them, um, was to find all of the papers that Liam Feddes and Barrett Zoff had written together. I think there was 11. Um, the model now can find most of them or all of them. We would always ask that question. And then another one, which the model actually can't answer anymore, probably for good reason, but the middle, finding the middle name of one of our co-workers. And then personally, I, I think I started using it pretty early on for Actually finding information for like product recommendations, travel, and I think actually quite a few people internally, we had kind of a streamlit playground that people would just use. A lot of people had found it and were using it. Sam told me he used it to buy a bunch of things. Every time it would go down, people would message us like, what happened? We need to use, um, the model, even when a previous version that honestly wasn't that good. So I think that was a good initial sign.

AI assessment note: “One of them, um, was to find all of the papers that Liam Feddes”

Q What sort of data are you using in order to actually identify a mine site or a potential site?

A Okay, so there's, there's a huge amount of data. Humans have been collecting data about the earth for, you know, for as long as humans have been looking at rocks, right? And there's an enormous amount of data Great deal of which is actually in the public domain, and the length scales are very different. Start with the global length scale. What can you know about the earth, the entire earth? Well, you can look at satellite imagery, uh, and you can look at satellite imagery in different colors, and so you can get a sense of, uh, of the rocks that are exposed at the surface, and there are data sets that tell you about the structure of, uh, of the continents, and the ancient continents that collided, and where the sort of ancient continental proto-continents were, and where those crashed into each other a long time ago and formed mountain ranges. You zoom in and you go to another length scale and you can fly airborne surveys with sensors on them that can detect the magnetic properties and the density and the electrical conductivity of the rocks. Go out and collect rock samples and measure what they're made out of. All the concentrations of different chemical elements and likewise for soil samples. And these are standard types of data that are used in the industry. And there's a huge number of these old data sets That are in the public domain. Uh, they are, most private companies ha…

AI assessment note: “satellite imagery”

Q Josh, can you give us a sense of just like the scale of the operation for Kobold today and like, you know, where you are looking, where you own land, where you're drilling, what, what you've discovered?

A Absolutely. So We, um, we operate exploration projects. So we have the, basically the company does two things. We, uh, we, we find places that are prospective for making discoveries. And then we, we go test our hypotheses by going and collecting data, collecting rock samples, flying airborne surveys, drilling holes to get samples of rock from below the ground. And we develop technology that we use for guiding our decision-making. So our exploration portfolio is more than 60 projects. And they're on four continents. They're in North America, Europe, Australia, and critically in Africa, targeting copper and lithium and nickel and cobalt and likely other commodities to come. And again, in all of these cases, we own the exploration rights, either ourselves or in combination with a joint venture partner, and we are operating the exploration programs. Almost all of these are pre-discovery opportunities. They're seeds we've planted. Any of them Could it become great ore deposits? And what we have in Zambia is, uh, is really an extraordinary deposit. It is the highest, uh, highest grade large copper deposit that is not yet a mine. The average of operating copper mines today, uh, is that the concentration of copper in the ore is, is about 0.6%. So if you mine a thousand kilograms of ore, not including the non-ore rocks all around it, There's six kilograms of copper in it that you can po…

AI assessment note: “our exploration portfolio is more than 60 projects. And they're on four continents.”

Q um, you went and did consulting, and you worked in oil and gas, you worked in private equity around it, and so that feels more relevant. Um, but this is like such a cool discovery of an interesting problem that you might go apply, uh, You know, decision making science and data too. Um, how did you decide that you wanted to go work on mining and like better exploration?

A So I've always been interested in the intersection of, of energy and technology. I, you know, I studied physics because just like grappling with hard questions. Uh, so I did a PhD in quantum computing. I've just been interested in physics because it's like, you know, I like working on hard problems. I like learning things. Um, but wanted to apply that to the most relevant things in our society today, which relate to our energy systems. And, and, you know, I went and worked with energy companies as a, as a management consultant, uh, with power companies and oil and gas companies and industrial companies who make power equipment, oil field equipment. And, and, uh, my co-founder Kurt House and I were doing private equity investment in oil and gas together. Uh, in the private equity firm whose, uh, whose leaders had sponsored his previous startup company. And we had become friends as graduate students at Harvard together. He was also studied physics and philosophy as an undergraduate and then applied math and earth sciences as a graduate student. And we did, uh, we, you know, we would read papers on energy topics and go to visit power plants and coal mines and things like that. And So we're already working the energy system and quite, quite interested in how the raw materials relate to the global economy. And we decided we didn't want to work on fossil fuels anymore. This is 2018. …

AI assessment note: “we decided we didn't want to work on fossil fuels anymore. This is 2018.”

Q Are you trying to offer anything that isn't comp?

A Yeah, we are. So one of the things that we realized is that the average labor marketplace has a 50 to one ratio of supply side relative to demand side, which means the average person that applies talks to their friend who also applied and neither of them got jobs. And it's almost just this like structural part of building labor marketplaces. The way to actually scale up the labor marketplace to have hundreds of millions of the smartest people in the world on the platform is to build all of these free tools such as AI mock interviews, AI career advice, Uh, you know, shareable profiles for people. All of the things that just create the most magical experience possible for consumers and give that away for free because it's powered by this monetization engine on the other side of the business. And so that's a very significant focus for us.

AI assessment note: “Yeah, we are. So one of the things that we realized is that”

Q So one thing that you all came out with today from Pi was what you call a hierarchical interactive robot or HiRobot. Um, can you tell us a little bit more about that?

A So this was a really fun project. There's two things that we're trying to look at here. One is that If you need to do like a longer horizon task, meaning a task that might take minutes to do, then if you just train a single policy to like output actions based on images, um, like if you're trying to make a sandwich and you train a policy that's just outputting the next motor command, uh, that might not do as well as something that's actually kind of thinking through the steps to accomplish that task. That was kind of the first component. That's where the hierarchy comes in. And the second component is A lot of the times when we train robot policies, we're just saying, like, we'll take our data, we'll annotate it and say, like, this is picking up the sponge. This is putting the bowl in the bin. This segment is, I don't know, folding the shirt. And then you get a policy that can, like, follow those basic commands of, like, fold the shirt or pick up the cup, those sorts of things. But at the end of the day, we, we don't want robots just to be able to do that. We want them to be able to interact with us where we can say, like, oh, I'm a vegetarian. Can you make me a sandwich? Oh, and I, like, I'm allergic to pickles. Maybe don't include those, um, and maybe also be able to interject in the middle and say, like, oh, hold off on the tomatoes or something. It's actually kind of a big g…

AI assessment note: “There's two things that we're trying to look at here. One is that”

Q of sensors besides just vision. So you have LiDAR and a few other things. As ways to augment the self-driving capabilities of a vehicle. Where do you think we are in terms of the sensors that we use in the context of robots? Is there anything missing? Is there anything we should add? Are there types of inputs or feedback that we need to incorporate that haven't been incorporated yet?

A So we've gotten very far just with vision, with RGB images even, uh, and we typically will have, uh, one or multiple external kind of what we call base cameras that are looking at the scene and also cameras mounted to each of the wrists of the robot. We can get very, very far with that. I would love if, like, skin, uh, if we could give our robot skin. Unfortunately, a lot of the tactile sensors that are out there Are either far less robust than skin, far more expensive, uh, or very, very low resolution. And so there's a lot of kind of challenges on the hardware side there. And we found that actually that mounting RGB cameras to the wrists ends up being very, very helpful and probably giving you a lot of the same information that tactile sensors can give you.

AI assessment note: “I would love if, like, skin, uh, if we could give our robot skin.”