Everything Jake Heller said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Heller: AI startups face mass extinction as enterprise pilots fail to convert
“A lot of those pilots are not converting to real revenue, and there's going to be a mass extinction event as a lot of pilot revenue, it's like, instead of ARR, it's like PRR, like Pilot Recurring Revenue, or something like that, or not even recurring, just Pil…”
Heller: AI application TAM is 1,000x larger when capturing labor budgets
“Today, the actual amount of money that we already know people and companies are willing to spend is the combined salaries of all the people they're currently paying to do the job. And that number is like a thousand X bigger. You pay 20 dollars a month to solve…”
Heller: Casetext built CoCounsel with plain Python, dismissing frameworks like LangChain
“And to be honest, a lot of the stuff that we built while building co-council was exactly like this. Every time you do this task, you're basically gonna take the same six or seven steps. And you don't need to have, frankly, like fucking Langchain or whatever. J…”
Heller: GPT technology is underhyped because GPT-4 operates at postgraduate level
“So this is gonna sound probably insane to most people who hear it, but I think the GPT technology is under hyped because what we see in GPT-IV specifically as a model, and I'm assuming the same will be true as new models come out in the state of the art advanc…”
Heller: Most AI app developers fail to run evaluations or study workflows
“If you just do these two things, you'll be like, 90% of your way there to building a better AI app than what most of the crap that's out there, right? Because most people never eval, and they never take the time to figure out how professionals really do the jo…”
Heller: AI startups with superior UI will lose without strong customer enablement
“Your product isn't just the pixels on the screen. It's not just what happens when you click this button. It's the human interactions with your support, customer success, with the founder it's training, it's, you know, everything around it. If you don't get tha…”
Heller: No single AI company will win entire knowledge work markets
“First of all, for some of these spaces, the market is so big because we're talking about like how much, how many trillions of dollars are being currently spent on like marketing professionals or support professionals or whatever. There's not going to be a sing…”
Heller: Targeting legal tech pre-LLMs was a mistake due to low software spend
“I actually feel like in some ways for us in the early days focusing on legal, Made sense for us because I knew legal, but also it's kind of a mistake because at the time the legal software industry Pre-LLMs is actually pretty small because it was like a fracti…”
Heller: AI defensibility comes from the brutal complexity of building it
“Just build it, and as soon as you build it, you'll see how fucking hard it was to build it, how many little pieces you have to build, how many data integrations, how many checks, how fine-tuned the prompts need to be, how you have to pick your models super wel…”
Heller: Casetext pivoted with $20M revenue and 100 employees for GPT-4
“We were like twenty million dollars in revenue. We were doing great. I had like a hundred people and we stopped everything that we were doing and said, we're going to build something totally new based on this new technology.”
Heller: Systematic prompt engineering can replace fine-tuning in AI apps
“What you'll find without any, these are for fine tuning, without any like technical fine tuning, you can go so far with just prompting. If you're being really careful about this, you will find that the AI gets things wrong predictably.”
Heller: Roles companies already outsource abroad are ideal targets for AI automation
“What are the kinds of roles that people are currently outsourcing, say, to another country? Right? If it's something that they're willing to do that for, then that's probably a pretty good target for what AI could take over.”
Heller: AI competition will drive legal service costs to pennies on the dollar
“I think at first you can start charging when the human's charging and then you'll have competitors and they'll come in and they'll charge a little bit less. And then other competitors will come and they'll charge a little bit less. And it's kind of beautiful, …”
Heller: Casetext poised to triple annual revenue after CoCounsel launch
“I mean, we were almost certainly going to triple our revenue in the course of this year, and that's after 10 years of building the revenue we had before, right?”
Heller: AI architecture will mirror cloud with base, tooling, and application layers
“To me, GPT and other models is like a base layer of capability, like cloud computing that a lot of companies, if not all companies will plug into to some degree. And there'll be another layer of technologies that help you get the most out of it. Langchain's a …”
Heller: Casetext received early access to GPT-4 in summer 2022
“Because we were so focused on large language models and were researching deeply in this space, we got really early access to GPT-IV. Like summer, 20, 22.”
Heller: 30% to 40% of Casetext Employees, Including Coders, Were Lawyers
“It was helpful for us that, that I was a lawyer, my co-founders were lawyers, 30 to 40% of my company, even the coders were lawyers, because we actually lived it.”
Heller: Casetext derived most evals from customer failures, not lab tests
“We've added much more evals at this point from real things that happened to real customers than the ones we came up with in the lab.”
Heller: Enterprise customers preferred predictable $6,000 annual seats over usage pricing
“When we asked our customers, they said, listen, I'd rather pay more, but make it like consistent throughout the year, then potentially pay less and pay per use. So our customers wanted to pay a 6000 dollars per seat. They want to proceed and they want to pay 6…”
Heller: Early enterprise law firm clients paid Casetext $50K–$150K
“They started paying us 50,000, a 100,000, a 150,000 dollars per client, and we got super excited about that.”
Heller: Casetext received GPT-4 access six months before release
“When we got the chance to work at GPT-IV, maybe about six or so months before it was publicly released,”