Dec 11, 2024 · 26m · saastr
5 Things That Are Working and 5 Things That Aren’t in B2B SaaS AI with Ironclad's CEO and a16z
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Ironclad CEO Jason Boehmig and Andreessen Horowitz Partner Seema Amble analyze the state of enterprise AI in B2B SaaS, outlining five proven strategies delivering commercial value and five critical pitfalls currently impeding software founders.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
Jason as informed peer 0.0 Guest teaching 0.0 Guest disagreement 0.5 Jason pushing back 0.0
speaking balance: gold is Jason, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 20:03 Skepticism of PLG orthodoxy
Jason rejects conventional SaaS wisdom about PLG, arguing founders should prioritize million-dollar enterprise deals over self-serve funnels.
Hardest push from Jason ▶ 13:47 Nuance on Founder Mode narrativeJason clarifies that while he embraces the core spirit of founder-led AI innovation, he disagrees with extreme interpretations of the founder mode trend.
Biggest teaching moment ▶ 12:44 Accounting benchmark accuracy dataSeema educates the audience on empirical benchmark disparities between generic base models (20%) and fine-tuned domain models (90%+) in professional accounting.
Jason holds their own ▶ 9:04 Main branch architecture validationJason demonstrates domain engineering expertise by explaining why staying on main branch LLMs with RAG beats maintaining costly custom-trained models.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Jason as informed peer | Guest teaching | Guest disagreement | Jason pushing back | Why |
|---|---|---|---|---|---|---|
| Speaker Introductions and Presentation Overview | 0 | 0 | 0 | 0 | Seema Amble and Jason Boehmig introduce themselves and their background at SaaStr, discussing AI-driven onboarding and implementation. Because this is a co-delivered conference presentation without an active interviewing host, host-side metrics remain at zero. | |
| What's Working: Automating Basic Manual Workflows and Data Entry | 0 | 0 | 0 | 0 | The presenters discuss automating basic manual tasks like transcription and form filling in SaaS. Jason shares historical context on Ironclad starting as an AI company in 2014 before the underlying models worked. | |
| What's Working: Staying on the Main Branch of Foundational Models | 0 | 0 | 0 | 0 | Jason and Seema explain why staying on the main branch of foundation models alongside RAG is outperforming full custom foundational builds for vertical applications. | |
| What's Working: Capitalizing on Vertical Data and Industry Context | 0 | 0 | 0 | 0 | Jason explains conceptual redlining in legal tech, while Seema shares benchmark data comparing standard base models against fine-tuned models in accounting. | |
| What's Working: Founder-Driven Execution and AI Incubation | 0 | 0 | 1 | 0 | Jason discusses applying 'founder mode' to AI incubation by sequestering five top engineers to prototype a standalone AI paralegal product. Mild pushback against certain extremes of founder mode dogma. | |
| What's Not Working: Broad Horizontal Approaches Over Narrow Solutions | 0 | 0 | 1 | 0 | The speakers discuss why broad horizontal AI tooling struggles against focused vertical solutions, warning that the industry is at the peak of inflated expectations. | |
| What's Not Working: Relying on Self-Serve PLG for Enterprise Sales | 0 | 0 | 2 | 0 | Jason gives a contrarian hot take on PLG, stating enterprise AI companies should pursue large repeatable deals rather than over-indexing on self-serve economics. | |
| What's Not Working: Unsupervised AI for Complex and Deterministic Tasks | 0 | 0 | 0 | 0 | Seema and Jason explain why human-in-the-loop oversight is essential for deterministic knowledge work like law and accounting, and detail emerging monetization models. |
Statements from this episode (20)
Amble: AI Onboarding and Templates Accelerated AI Presentation Software Growth
“There's a whole set of presentation software, AI-driven presentation software that's been growing really quickly in the last year or two. And this category actually I wouldn't say struggled, but it was growing much more slowly. It was harder to get folks to ge…”
Boehmig: Ironclad Uses AI to Ingest Contracts and Automate Implementation
“What we're seeing early progress on is this ability of AI to help actually discover preferences, ingest a bunch of contracts, understand what the legal team we're working with likes to do, and then actually help implement that in the software so that when they…”
Amble: Vertical SaaS Startups Scale by Automating Manual Workflows With AI
“On the vertical SaaS side, a lot of, we're seeing a whole set of new companies that are popping up around these sort of basic manual work processes, because if you're a vertical provider, so if you're providing software to vet clinics, for example, There's a l…”
Boehmig: Ironclad's 2014 AI Contract Extraction Vision Failed Then, Works Today
“We started Ironclad in 2014 as ironclad.ai. And the basic idea around this was companies have lots of contracts. They don't really know what's in them. What if we used AI to read the contract and build a database? That turns out that didn't actually work in 20…”
Boehmig: Sticking to Main-Branch Foundation Models Beats Custom Model Training
“Staying on the main branch, staying on the core capabilities and supplementing it with fine tuning and additional techniques like RAG seems to be the approach that's winning right now.”
Amble: Most a16z SaaS Portfolio Companies Choose Main-Branch Foundation Models
“Making your life easier and getting most of the benefits of AI by staying on the main branch is where we are right now. And so that's like what we're seeing resonating with more of our companies.”
Boehmig: Ironclad Customers See Tens of Millions in AI Business Value
“We're working with customers who are seeing tens of millions of dollars in attributable business value to speeding up attorney workflow, doing things in house that they used to have to send out out house.”
Amble: Fine-Tuned GPT-4o Reached 90%+ Accuracy vs 20% Base in Accounting
“I met an accounting AI company a couple of weeks ago that had said, They tested GPT-IV-O against their benchmarks, and it was like, 20% accurate. They then tested Claude, and it was like, 50 to 80%, and then a fine-tuned version of GPT-IV-O, they were getting …”
Boehmig: Ironclad Generates Nine Figures in Revenue
“So we're, we have this like enterprise product, nine figures of revenue, growing great, good clip, a lot of the company oriented around that.”
Amble: Standalone AI Products Require Direct Founder-Driven Energy to Differentiate
“If you're launching an AI product, a standalone product, or it's not just like the feature top up saying summarization or autocomplete, but it's really a standalone product. You really do need that like founder level engagement and drive to push the product fo…”
Amble: AI-First SaaS Startups Are Narrowing to Specific Verticals
“What we are seeing is a lot of companies who are building AI first SaaS applications start to narrow down either on a use case or a market versus being more horizontal.”
Boehmig: AI Is at Peak Hype and Heading Into a Trough
“I do think we are at the peak of inflated lofty expectations in AI, and there's a, for good reason, there's a lot of excitement, but I think the trough is coming, and what we gotta do to get out of the trough is find the repeatable value in AI, and it, I think…”
Boehmig: 60% of Ironclad Enterprise Customers Have Adopted AI
“We have 60% of our customers that have adopted AI.”
Boehmig: Selling Repeatable $1M ARR Deals Beats Optimizing Enterprise PLG
“I think my big learning from this is not everything needs to be a PLG product. And if I could go give myself advice three years ago, it would be go figure out a way to sell more million ARR repeatable deals. And that's your path to efficiency, as opposed to tr…”
Amble: LLMs Lack Planning Abilities Needed for End-to-End Automation
“The LLMs are not good at planning, right? They're good at predicting the next step. And as part of that, you can't rely on LLMs to do the end-to-end Processing yet.”
Boehmig: Human-AI Collaboration Will Beat Pure Machine Automation
“This combination of human and machine will beat the machine only.”
Amble: Many Failing Enterprise AI Use Cases Will Work in One Year
“Again, I check it in a year. I'm sure there'll be many more use cases that weren't working now that aren't working out that will be then, but yeah.”
Boehmig: Unpredictable 5% Error Rates Make Unconstrained AI Dangerous in Contracts
“Especially for knowledge work, you don't want five percent of your contract work to be wrong, especially if you don't know what five percent might be wrong. Huge cost to that. And having the right human checkpoints there is still the best way to do things, and…”