The Exchanges

Q missed the whole SaaS wave, but starting off in AI, it was all chat based. Now we've completely moved to the agentic era, which is like a thousand times more cost competitive. You need compute power, memory, All these sorts of things that are reorganizing how companies are being built and of course the stock market too. So how are you thinking through the agentic layer and building that out?

A Yeah, so I think this was like a huge transition we've gone through in like the past six months where exactly like you said, the original product we built was kind of this chat based co-pilot for lawyers and then we built all this product around it that Let it work at a law firm on client matters and all these things. And then maybe six months ago when these coding models started getting really good and you could do kind of things like cloud code and codex and that started working really well in the command line, we started building a bunch of the infrastructure of, you know, how do you get these agents that were running in your command line and could execute all these tools and work really well and how to move that into the cloud. And I think now you're seeing this with kind of these managed agent solutions. Um, and to your point, it's like now you need sandboxes. The models use way more tokens. Um, we have queries that like, we have simple like assistant type queries where you say, you know, draft me a document that a single query can cost 20 dollars. We have, um, like a review product where you can upload a 100,000 contracts and ask the models to review them, and some of those can cost 20,000 dollars. And so it is just getting Incredibly expensive, and I think in parallel what we're seeing is I think even two years ago the models were so good with coding that most programmer…

AI assessment note: “we started building a bunch of the infrastructure of, you know, how do you get these agents”

Q Benchmarking makes things a lot more competitive and people love that. I mean, AI is like the most competitive world ever right now. And so I'm really curious on that standpoint, like, how are you then like, where does this, where does this lead to next?

A Yeah. So, so I think the interesting thing and kind of to your point is, okay, we released this benchmark and now we're going to have all of the labs and everyone else building models competing on it. And I think there's obviously some risk of, oh, what if some lab providers models are the best, then like, why do they need Harvey to do this is, is maybe one of the potential risks. But what we've seen is actually every different model is good at something different. And so with the initial results we saw, Anthropics models are quite strong, but there's areas where a 5.5 is better. There's some areas where open source is better. And increasingly, it's not just which model is the best. It's which model can solve the task at the lowest price point, because as these models get better, there's kind of intelligent saturation where it's like, okay, for some simple tasks, I actually want to know that this open source model is good enough. And so a lot of the value that we're trying to provide to law firms is You can't just use one model to solve all these tasks because it's getting too expensive. So how do you think about all the tasks that your law firm does and which model you should use from which provider to solve those tasks?

AI assessment note: “increasingly, it's not just which model is the best. It's which model can solve”

Q What do you think the biggest question is people are not asking?

A I feel like the, probably the big, like, misconception right now is I don't think people realize how expensive this is going to get, and I don't think people realize how difficult it is going to be for customers to deal with that. Like, I think when I talk with most people, they think, oh, just move to consumption pricing and that will solve all the problems, but I think there's going to be this very interesting Dynamic where there's kind of a couple ways to price these things. And I think most VCs, what they want to see is like, can you price the work, right? Like, can you sell the value of the work you're selling? Uh, and then the, like, that's one end of the extreme, and then the other end extreme is just like price by tokens. And I think the problem you run into of pricing the work is actually the same problem that law firms run into when they try to do fixed fee pricing. Right? They're trying to say, hey, here's the fixed rate of this cost. And I think there's going to be this great irony where when we started the company, one of the questions we got asked the most was, you know, what's going to happen with the billable hour? And I think most people's assumption is just, oh, everything's going to move to fixed fee, so then these law firms can protect their margins. And I think something people don't appreciate about the billable hour and why it's such a good mechanism is, …

AI assessment note: “I don't think people realize how expensive this is going to get”

Q I know you've answered this before, but your research partners are your biggest competitors. OpenAI, Anthropic, NVIDIA, we have base 10. There's like a handful of across categories, right? So why open source it? Doesn't that pose a risk to you?

A Yeah, I think we, we think of all of these providers as there's obviously overlap Um, with the labs and, you know, Anthropic and OpenAI have done kind of like Cloud for Legal, Codex for Legal, but I think the problem that we're solving is different. Like, I think a lot of the products that they're building are individual productivity, and the products we're trying to build are organizational productivity for law firms or large in-house departments, and so when we think about open sourcing, I think there's kind of the initial point that I made of, like, most of these law firms and enterprises Can't rely on a single lab provider. So like there's a big risk if you're a law firm that if you just use Anthropic or you just use OpenAI, um, you run into conflict risk. So imagine you're using only Anthropics models as a law firm and you want to represent OpenAI. OpenAI is not going to let you send their sensitive legal data to Anthropics models. And so as a law firm, you need to support all the different models if you want to represent OpenAI and Google and Microsoft and all these providers. So there's the conflict risk. There's like platform risk. Like what happens if you pick one provider and they run out of compute, their models fall behind. And so you need to kind of abstract all these things away. And so we think of the providers similar to cloud. And I think you see the same thing…

AI assessment note: “the problem that we're solving is different... organizational productivity for law firms”

Q So speaking on your background, you were at Google Brain, DeepMind, and Meta. What were the biggest fundamental lessons that you learned from each of them that are driving the decisions you're making?

A Yeah, I think especially Google Brain and DeepMind when I was there, this was kind of in 2016, 2017, and so right when deep learning was starting to take off, and I think it was, it was a really cool experience seeing both of those labs because they had kind of opposite strategies of how they approached research, which I think both were correct, but led to kind of different ways of executing on kind of the vision of Of where they thought AI was going. And so when I was at Brain, it was very bottoms up. It was, I mean, both had some of the smartest AI researchers in the world, but Brain, the approach was, you know, let's get all these smart people, give them a bunch of compute, and then kind of let them do their own projects. And I think part of the outcome there is you had someone like Noam Shazir and Ashish Viswani and kind of the rest of the folks that invented transformers. And so that approach worked. And then DeepMind was much more top down where Demis just had this vision of, okay, we're going to create AGI. Here's all the things that I think are required. And so they kind of had this tech tree of how they were going to do it. And then let's have all these teams working on research projects that would solve all these milestones to get us to AGI. Um, and I think our approach is a bit more inspired by the DeepMind one, just because I think because we're in a vertical, the e…

AI assessment note: “our approach is a bit more inspired by the DeepMind one”

Q Given learning from all of them, what are the key traits that you look for in new hires?

A Yeah. Key traits in new hires. I think the biggest is Like, are you just obsessed with the topic? Like the best hires that we've made, it's usually really easy to talk to them about what we're doing because they know so much already. And so I remember like with Nico, Daniel, Spencer, like a lot of our like early hires, um, when I would just pitch them, here's what we're doing. It just, they immediately were like, That totally makes sense. Here's a bunch of spinoff ideas, and we could just, like, talk forever about this. And so I think that's something that I always look for, of, like, usually if I can have these conversations with someone where we're just building off each other, like, that's always been a strong indicator. Like, I think, I think it's somewhat unique, but I think Winston and I's, like, hit rate on, like, executive hires is, like, insanely high, and it's usually, like, if we can both have a conversation like that with someone, And they're, you know, background, everything's a good fit. We're like, okay, this person is, is gonna work out, and I think that's been, like, pretty accurate. Um, and you can usually just feel someone who's, like, insanely passionate or, like, obsessed with a topic versus, like, someone who's, like, doesn't know what's going on, and so I think that's kind of one of the biggest, like, indicators.

AI assessment note: “I think the biggest is Like, are you just obsessed with the topic?”