Heller: Most AI app developers fail to run evaluations or study workflows
Jake Heller · From Idea to $650M Exit: Lessons in Building AI Startups · Y Combinator · Oct 28, 2025 · at 23:17
Jake Heller, co-founder and CEO of Casetext, emphasizes the importance of systematic workflow decomposition and evaluation in building production-ready AI products.
0:00 / 0:12exact quote · 12.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If you just do these two things, you'll be like, 90% of your way there to building a better AI app than what most of the crap that's out there, right? Because most people never eval, and they never take the time to figure out how professionals really do the job.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jake Heller
Heller: AI startups face mass extinction as enterprise pilots fail to convert
“A lot of those pilots are not converting to real revenue, and there's going to be a mass extinction event as a lot of pilot revenue, it's like, instead of ARR, it's like PRR, like Pilot Recurring Revenue, or something like that, or not even recurring, just Pil…”
Heller: AI application TAM is 1,000x larger when capturing labor budgets
“Today, the actual amount of money that we already know people and companies are willing to spend is the combined salaries of all the people they're currently paying to do the job. And that number is like a thousand X bigger. You pay 20 dollars a month to solve…”
Heller: Casetext built CoCounsel with plain Python, dismissing frameworks like LangChain
“And to be honest, a lot of the stuff that we built while building co-council was exactly like this. Every time you do this task, you're basically gonna take the same six or seven steps. And you don't need to have, frankly, like fucking Langchain or whatever. J…”
Heller: GPT technology is underhyped because GPT-4 operates at postgraduate level
“So this is gonna sound probably insane to most people who hear it, but I think the GPT technology is under hyped because what we see in GPT-IV specifically as a model, and I'm assuming the same will be true as new models come out in the state of the art advanc…”
Heller: AI startups with superior UI will lose without strong customer enablement
“Your product isn't just the pixels on the screen. It's not just what happens when you click this button. It's the human interactions with your support, customer success, with the founder it's training, it's, you know, everything around it. If you don't get tha…”
Heller: No single AI company will win entire knowledge work markets
“First of all, for some of these spaces, the market is so big because we're talking about like how much, how many trillions of dollars are being currently spent on like marketing professionals or support professionals or whatever. There's not going to be a sing…”