The Exchanges

Q fail and, you know, where doesn't it? Were there specific things that Palantir did early on to On board people to sort of this new way of thinking or have people play around with the models in specific ways? I mean, you mentioned the hackathon. I'm just sort of curious how this all got started and, you know, how you now incorporate it into how people think about these problems.

A Yeah, we, we've made it a huge organizational focus really to experiment and play with these things. Like, so how could you bring this to your own, um, area of the product? But as importantly, like, how do we build this into our tool chain? So, hey, we are doing incident response on our stacks. Can let's, let's have, like, let's build a copilot for ourselves to go manage that more efficiently. And so by trying to solve your own problems with it, you get much stronger intuition of like where it's, Amazing and where it falls off a cliff and how you have to think about that as you, as you build it. So aggressively adopting it to drive our own productivity has been one dimension of it.

AI assessment note: “we've made it a huge organizational focus really to experiment and play with these things”

Q And when I look at the implications of LLMs and generative AI to healthcare, there's so much low-hanging fruit because it's such a big people-intensive services industry. It'd be great to just hear your viewpoint in terms of how you work with some of these healthcare customers and what you think this coming wave of AI will do. Like, what are the areas that will be most impacted by that?

A Yeah. Healthcare is roughly a third of our business. It's, it's certainly, I mean, it's probably one of the fastest growing parts of our business as well. Uh, and, and we do that, uh, you know, in, in a number of countries. So the NHS in the UK and multiple hospital systems in the US and across both kind of dimensions of clinical care and operational care, like the hospital operations. And I think that's relevant because the pace of adoption for, for these will vary and kind of the challenges you solve for the use cases with LLMs is different between them. I think the operational context is, is very obvious in the sense that it's just like operating any institution, really. You, you have kind of supply demand. You have labor inputs to that. You're trying to manage that so that you can deliver the product, the care that you actually have. And there it fits very cleanly to how will we help, you know, auto companies get better at what they're doing, or how will we help manufacturers or energy companies? And, and there, I think probably the archetypal pattern that I see across all industries is something like you today have something, if you squint at it, it looks like an alert inbox where, uh, you know, state machine is essentially saying, here's an exception or something that I need someone to think about. And the human kind of, you, then you, so many exceptions, I need some help…

AI assessment note: “turning that from a place where I'm surfacing alerts to a human to I'm surfacing solutions”

Q Yeah. And then you joined, as you mentioned, as employee number 13, the first business hire, et cetera, and you've had a variety of roles over time. Could you explain a little bit how your role has changed over the lifetime of the company?

A I think the, the kind of common thread through it all is just doing what you need to do, which I know sounds like a banality, but really when it, when it first started, I like wrote our first kind of candidate management system. What are you doing as a business hire before you have really a product here? And I was, I was, uh, an aggressive QA tester, you could say, but the, the real initial contribution was what we call forward deployed engineering. It comes from an insight that Alex kind of had around like, well, you know, he, he, he muses, Why are French restaurants so good? Well, maybe one theory is that the wait staff is actually part of the kitchen staff there. You know, it's, it's not that they have like deep context and understanding of the food. And so the, the Ford deployed engineering idea was that the people who are going to be interacting with customers in the field, we're going to be computer scientists. You could actually understand, uh, what does the product do today? What does it need to do today? How is it under what conditions is it going to work? And how do you kind of Create this hybrid role that's product management, customer success, and engineering all in one. And that, that's really the team that I first built up. And then as we went from Gotham and Foundry and now AIP, there, there's a lot to, to do there. There it's like, whether it's interacting more …

AI assessment note: “when it first started, I like wrote our first kind of candidate management system”

Q perception of the company is that very early on, a lot of the earliest customers were intelligence and defense, and then it kind of broadened from there. Is that a correct assessment, and was that intentional at the time, or was it just you found that there's a pocket of customers that really cared about your product and, you know, were a good fit for what you were doing initially?

A Well, yeah, we founded the company to, to work with intelligence and defense organizations. And, and really, I think, um, we expanded almost reluctantly, you know, I think it was like, 2010 or 20 11 where we started working, uh, with our first commercial customer. But really what we realized was that it took something as sexy as James Bond to motivate engineers to work on a problem as boring as data integration. But this sort of, we had our own ideas of what would be valuable in these spaces and we built software for it. But all of those ideas kind of presupposed That the data was integrated. And, and, you know, it's kind of, I think the kind of popular view is like, this is a boring and solved problem, but I think it might be kind of a boring and highly unsolved problem that people are kind of like duct taping together everywhere they go. And so by productizing a solution to that, we kind of expanded our market and, and the, what we could sorts of problems in the world that we could go after. Apollo is quite an interesting platform as well. So like we really originally built Apollo for ourselves. If you think about our customers, The, we're deploying in air-gapped environments. So how do you deploy modern software when, you know, you can't see ICD to the target? We had to build this entire infrastructure that allowed us, you know, our software, it's modern software. We have 55…

AI assessment note: “we founded the company to, to work with intelligence and defense organizations.”

Q What, what are some of those components that you think are the tool chain that you need to sort of bottle the stochastic genie? And I love that phrase by the way. I think it's a really good way to put it.

A So you're, you're probably going to need everything you kind of need with the dev tool chain, but you're going to have to adjust it for the fact that it's stochastic. So you even see it like people call it eval and not unit tests, but you're going to need like, how many unit tests do you need? If you're going to write an LLM backed function and it's a stochastic genie, how, how many times does it need to execute before you have confidence that it's going to do what you want? And then, so then you can think about that. That's like day zero. Ok, so I, I build this thing. How do I think about it? But what sort of telemetry and production log data do I need? Uh, and, and how, how often am I going to be looking at those traces? And it's like, I might even be writing unit tests against my traces. I guess you could call that like a health check, right? And like, there's going to be, um, a lot more emphasis that, that you're going to need there as an engineer, as you think about using this. And then there's going to have to be some calibration on the use case. The best use case is going to be ones where When the LLM gets it right, there's massive upside. And when it doesn't, it's a no op, right? Uh, and, and so picking those ones I think are going to be quite important as you build and tune the specific applications of these.

AI assessment note: “people call it eval and not unit tests... telemetry and production log data”

Q programmatic agent driven world where five years from now or in years from now, you just have agents that represent you as a user with a specific task interacting with other agents or APIs. To your point, you really minimize the UI dramatically. Do you think that's, that's the most likely future or how do you sort of think about where all this stuff is heading? From a UI perspective.

A It's, it's hard to see so far in the future on this, but what I think it definitely does, when I think about the integration layer, like when I, like you look at like the gorillas paper and, and can, you can teach, you know, can you fine tune an LLM to basically tell you what API to call with what parameters? Like, yes, it turns out. And so, okay, so if that's true, what does system integration look like in the future? That's going to be quite different. So then I think it allows you to create more single panes of glass that are actually truly integrated. Which is incredibly hard right now. I think there's some subtle and interesting benefits. It's like one of the consequences of a hack we had a number of months ago was that I had an engineer who could build a feature, uh, in a couple hours that we had previously scoped. It was on the roadmap. It was a feature that was going to take like two months and two people. And it's just simply because the amount of UI that was involved was so intensive. You just replace the UI with language, the whole thing changes. So like, that's, that's, that's one way of thinking about, okay, well, what sort of UI are you not building today? That you actually don't even have to build today and that you, you probably have the tools, the primitives in the back end or the application that you can now surface. So I think that's, that's an interesting pl…

AI assessment note: “You just replace the UI with language, the whole thing changes.”