why aren't all 28 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Opinion
Weinberg: Standard AI benchmarks are useless for evaluating legal AI
“Most benchmarks are completely useless for us, right? And so we'll get a model, you know, someone will give us early access to a model and they'll say it's way better on all of these benchmarks and we'll respond. It actually isn't like, it's not used as useful…”
Assertion Supported
Pereyra: Law firms are acting as Harvey implementation partners for clients
“Law firms are starting to do this for their in-house clients. So they're starting to go and take Harvey and go to their clients and say, Hey, buy Harvey and we'll help you build all the workflows and implement it because we have the scale and the expertise to …”
Insight
Pereyra: Building a tech-enabled law firm requires running two incompatible companies
“The big challenge that they ran into was, you're essentially just building two different companies, right? You're building a law firm, And you're building a tech company, and it's already really hard to, like, build product engineering, do AI, scale sales, and…”
Insight
Weinberg: Earning enterprise trust requires targeting the hardest customers first
“We found that no matter what, you eventually have to get the trust of the industry. And the best way to do that is to actually go after the hardest people first.”
Insight
Weinberg: Vertical AI founders should evaluate the 'price per token'
“So one thing that I look at is you can look at this by industry and you can look at it by task, which is how expensive is the token? And I, I've come up with like different ways to call this, but basically if you look at something like that share purchase agre…”
Disclosure
Weinberg: Harvey is helping law firms build and sell specialized software
“We were starting to work with firms and basically take, you know, the special things that they do and the way that they practice law and turn it into their specialized system in Harvey. And then they go and sell that to their clients.”
Insight
Weinberg: Founders should do every role themselves before hiring for it
“One of the lessons that I definitely want to keep is I do think you should do every single role For a certain amount of time before you hire for it. Almost all of my mishires were because I did not understand what that role was.”
Insight
Weinberg: Productivity AI needs lower quality thresholds if work is verifiable
“On the productivity side, the minimum viable quality of that output can be lower because you're selling seats, And at the end of the day, there are multiple people reviewing it, right? And so what you want to do in that state is have just show your work, right…”
Insight
Weinberg: Enterprise AI requires pairing broad productivity tools with chained vertical workflows
“You need to build productivity tools, and what I mean by this is
Things that are useful for the highest amount of seats, right?
And then you also need to build things that are streamlined vertical workflow from start to finish, right?
And what you can do is…”
Insight
Pereyra: In Legal AI, the RL Environment Is a Client Matter
“And in legal, that RL environment is a client matter. So you have all of the context of a fund formation, an acquisition, a litigation, and the models are starting to learn. Let me go in the document management system and see if I can find this, go in the data…”
Insight
Gabe Pereyra: Client conflicts limit AI law firms compared to software providers
“Solving that equation at scale is a much bigger opportunity than if you build a single law firm Because you get conflicted out. You can't scale this.”
Assertion Supported
Pereyra: Harvey signed Walmart and works with AT&T and Fortune 500 clients
“So we recently announced we signed Walmart. We're working with AT&T, a bunch of these fortune, 500 large private equity firms, global 2000, kind of the largest consumers of legal services.”
Insight
Pereyra: Enterprise AI bottlenecks are orchestration and governance, not model intelligence
“When you get to that scale, a lot of the problems we're solving are not just model intelligence problems. They are these orchestration, governance, and kind of all of the enterprise product problems that you run into at scale.”
Insight
Pereyra: Legal and Coding See AI Traction Because Both Workflows Are Unstructured
“Legal is so difficult is the workflows aren't structured. So the same way with programming, it's really hard until these models to build tools for programmers. You basically just had an ID and then programmers did stuff in all the different languages, but you …”
Insight
Pereyra: Complex legal drafting lacks binary verifiability for AI reward functions
“For something like generate this merger agreement, it's really hard to just give some binary like this is good or this is bad. And I think this has been like a big research problem, like with all the labs we work with, and also internally, there is just this o…”
Insight
Pereyra: Narrow AI point solutions fail to leverage generalist model power
“If you had built something where it's like all this does is like check that your Python code doesn't have bugs, which you could have done better with 3.5, like you wouldn't have built something like cursor. And the intuition was just these models can help you …”
Assertion Not checkable as stated
Pereyra: Fortune 500 legal departments lack standard systems compared to law firms
“When we start working with the Walmarts, the very large banks, the Fortune 500, they're much less standardized than these law firms, and so there is just this massive amount of work where we go to a large bank and they say, we don't have any document managemen…”
Insight
Pereyra: Legal AI Succeeded Early by Focusing on Document Upload and Citations
“I think it was finding the right form factor. And I think in legal, it was maybe a bit more obvious where the initial form factor was essentially like the initial feature we built that none of the products had at the time was upload a document and do something…”
Insight
Pereyra: Legal AI requires expert reasoning traces, not just public filings
“Like all you get from these public mergers is like an SEC filing. And so you do see the final result. But most of the value or what you need, I think, to eventually improve these models is the decision making process the same way you need these reasoning trace…”
Insight
Weinberg: Senior employees protect positions, making critical feedback seekers rare leadership signals
“There are a few folks that will come up to me and basically just not ask for positive feedback and just say, what did I get? Like, what do I need to improve? That is a massive signal because I think that something that happens as you get more senior in your ca…”
Insight
Weinberg: Legal AI workflows transfer effectively to tax and finance
“There's so many areas where legal is kind of the tip of the spear and then you can actually just kind of paralyze the same things that you're building and tweak them. And it works really well in other industries.”
Insight
Weinberg: Vertical AI evaluation requires senior domain experts, not junior staff
“And the reality is you have to hire very good lawyers who can actually evaluate these systems, and the same within tax and these other areas, and they can't be too junior, because if they were too junior and they were able to eval it, they would be senior, rig…”
Prediction Not checkable as stated
Weinberg: AI will allow junior lawyers to do strategic work earlier
“And so what I think will end up happening is the timeline will compress. So you will start being able to actually do the high level strategic work and interact with clients, which is what people really want to do earlier on in your career.”
Assertion Supported
Guo: Harvey has raised over $500M from OpenAI, Sequoia, Kleiner, and others
“They've now raised more than five hundred million dollars from investors such as OpenAI, Sequoia, Kleiner Perkins, GV, Bloodgill, and me.”
Assertion Partly supported
Guo: Harvey serves over 250 clients and generates $50M+ ARR
“You guys now serve, you know, more than 250 clients, more than fifty million in ARR.”
Prediction Not checkable as stated
Guo: AI agents writing executable code is a powerful new paradigm
“There's been very recently more attention or just more understanding of how powerful it is to have agents that in some way write executable code. Right. Because you can programmatically use many more tools you can call APIs. And I think if that is do a task th…”
Assertion Supported
Pereyra: Legal is a $1T market while professional services is $3T-$5T
“And I think to your point, the scope of this, like legal is a trillion professional services is something like three to five trillion.”
Assertion Not checkable as stated
Pereyra: Harvey reaches nearly 1,000 customers and 500 employees
“We're almost at a thousand customers, 500 employees. Started about just over three and a half years ago, and so been kind of scaling quickly since then”