May 29, 2025 · 48m · mad
AI That Ends Busy Work — Hebbia CEO on “Agent Employees”
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast recorded live at Data-Driven NYC, Hebbia Founder and CEO George Sivulka joins host Matt Turck to explore the future of enterprise AI, detailing how AI is evolving from basic chatbots into autonomous agent employees that reshape organizational workflows across finance, law, and corporate knowledge work.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 19.2% of the talking time here. How this is scored →
Matt as informed peer 3.7 Guest teaching 5.4 Guest disagreement 2.3 Matt pushing back 2.0
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 21:01 Rejecting vertical AI as VC fallacy
George forcefully rejects the common venture capital thesis of vertical AI, calling verticalization a massive fallacy and insisting generalization wins every time.
Hardest push from Matt ▶ 13:44 Reality check on GDP agent replacement claimsMatt refuses to accept George's aggressive timeline for agent enterprise deployment without pushing for a reality check on overhyped capabilities versus real-world adoption.
Biggest teaching moment ▶ 30:31 Abandoning a world-record re-ranker for LLM computeGeorge educates Matt on why classical search metrics fail for complex reasoning, revealing that Hebbia shelved its record-breaking re-ranker model in favor of heavy LLM compute loops.
Matt holds his own ▶ 30:22 Probing classical RAG re-ranker componentsMatt shows strong technical grounding in AI retrieval infrastructure by explicitly probing whether Hebbia still relies on classical re-rankers like ColBERT.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Stage Reconnection and Hebbia's Elevator Pitch | 3 | 3 | 1 | 1 | Matt warmly reconnects with George, recalling their interview two years prior while asking for an updated elevator pitch. George explains Hebbia's evolution into a horizontal workplace AI platform beyond initial financial and legal verticals. | |
| Leaving Stanford PhD for GPT-3 | 2 | 4 | 2 | 1 | Matt playfully teases George about his young age and early Stanford graduation before asking about leaving his PhD. George shares his lightbulb moment using GPT-3 in June 2020, realizing it was a meta-learner that surpassed his research trajectory. | |
| Hebbia's Trajectory, Scale, and Page Processing | 2 | 5 | 2 | 1 | Matt asks for company metrics and scale. George jokingly references VC confidentiality constraints before revealing that Hebbia went from processing 100 million pages last year to an expected 4 to 5 billion pages this year. | |
| The Concept of the Agent Employee | 3 | 6 | 1 | 1 | Matt introduces George's recent blog post on the 'agent employee'. George articulates how organizational design is shifting to treat AI agents as distinct nodes in org charts with emails, Slacks, and dedicated workflows. | |
| Prompting as Management and Enterprise Adoption Curves | 4 | 5 | 2 | 4 | Matt presses for a reality check on bold claims that AI agents will contribute more to GDP than humans within a decade. George acknowledges enterprise adoption latency using the transition from cash to credit cards as an analogy. | |
| Test-Time Compute and Extending Context Windows | 5 | 6 | 1 | 3 | Matt drills down into current AI research, specifically double-clicking on how to artificially extend LLM context windows. George explains inference-time compute scaling and how Hebbia pioneered multi-call runtime infrastructure. | |
| Matrix Architecture and Multi-Agent Grid Workflows | 4 | 7 | 6 | 2 | George directly attacks standard venture consensus by declaring vertical AI a 'massive fallacy'. He argues forcefully that generalization always beats specialization because top experts pull insights from outside their core domains. | |
| Spreadsheets as the True Paradigm for Knowledge Work | 3 | 6 | 2 | 2 | Matt highlights George's skepticism toward chatbot UIs. George compares chatbots to TI-84 calculators, asserting that flexible spreadsheets are the true paradigm through which knowledge workers operate. | |
| Technical Deep Dive: Ingestion, Indexing, and ISD | 6 | 7 | 2 | 3 | Matt initiates a technical deep dive into ingestion, indexing, and ISD vs standard RAG architecture. George reveals that Hebbia developed a world-record accurate re-ranker but discarded it because keyword/vector retrieval fails for deep agent reasoning. | |
| Multi-Model Strategy and Maximizer Router | 4 | 6 | 2 | 2 | Matt asks about multi-model usage and system scaling tricks. George explains 'Maximizer', an air-traffic-controller system that routes requests to achieve theoretical maximum throughput across 250 billion monthly LLM calls. | |
| Addressing Hallucinations and the Cost of Accuracy | 4 | 6 | 6 | 3 | Matt brings up model hallucinations in high-stakes industries like finance and law. George forcefully rejects the concern as 'old news' and 'fugazi', claiming models are already vastly superior to human workers when provided proper context. | |
| The State of AI Innovation and Startup Alpha | 5 | 6 | 5 | 3 | Matt prompts George on whether fundamental AI research is slowing down. George gives a contrarian response that 'alpha is gone' for starting new AI companies and criticizes traditional SaaS GTM playbooks in favor of hiring domain consultants. | |
| Building Value Cases and Calculating ROI for Enterprise AI | 4 | 5 | 1 | 2 | Matt asks how buyers evaluate ROI and price justifications for enterprise AI. George explains that while cost savings exist, the primary value driver is revenue expansion by giving workers unlimited expert analysis capacity. | |
| The Impact of AI on Junior Roles and Knowledge Work | 4 | 6 | 2 | 2 | Matt asks if clients are reducing junior analyst hires and inquires how George leads as a young founder. George uses Morgan Stanley's historical creation of the analyst role during the computer era to argue AI will evolve junior work rather than destroy it. | |
| Future Outlook: From Chatbots to Agentic AI Applications | 2 | 3 | 0 | 0 | Matt asks for a 3-year outlook to conclude the interview. George shares his core aspiration of transitioning users worldwide from simple chatbots to agentic, value-creating applications. |
Statements from this episode (27)
Few horizontal general-purpose workplace AI products exist today
“And the idea is that there's lots of consumer AI products, products where you can go and, you know, talk about your sushi restaurant and wine in San Francisco, but there's actually not very horizontal general purpose workplace AI products.”
Hebbia requires its 100 NYC employees to work in-office 5-6 days
“So we built a team of a hundred amazing incredibly smart folks all five days a week, sometimes six in office in New York city.”
Hebbia aims to scale to 300-400 employees by the end of 2025
“And so we were opening SF and we've already opened a London office which is incredibly exciting with goals to end the year at 300 to 400 employees.”
Hebbia is on track to process five billion document pages in 2025
“This year, we're already on track to process around four to five billion pages.”
Hebbia is deployed at 40% to 50% of top asset managers
“And now we're deployed at, I think, between 40 and 50% of the world's largest asset managers.”
Hybrid teams of humans and AI agents will dominate organizational structures
“You're going to start to have, you know, fully human organizations. Actually fully AI organizations like the one person billion dollar startup or, you know, these things that are effectively just APIs. And you'll actually have, and this will be the most common…”
Prompting AI agents will soon become synonymous with management
“As agents are rolled out, you'll actually start to see you know, people that are really good at prompting, really good at defining a process, be the best managers, and actually be the best at extending whatever their agenda is in the organization, or making th…”
Enterprises will still be deploying AI in chatbot form in 10 years
“We will still be deploying AI. In its current form as a chatbot in 10 years.”
Hebbia pioneered inference-time compute scaling two years ago
“And actually the whole scaling at inference paradigm was pioneered at Hebbia. So our early matrix product two years ago, we're one of the first people to say, hey, you get way better accuracy from using more large language model calls at runtime.”
Elongating context windows is the primary bottleneck in artificial intelligence
“And I think the number one problem in all of AI is elongating the context window.”
Generalized AI software will defeat vertical AI specialization
“One of the things that is a massive fallacy in AI applications today is verticalization. As paramount. All of the VCs, a lot of entrepreneurs as well, believe, because it's been true for the last 20 years, that building a very verticalized piece of software is…”
Chatbots are like calculators, inadequate for serious enterprise knowledge work
“Chatbots are, in my eyes, like the TI-eighty-four, or like the HP-twelve-c, like a calculator. It's a one-off, you know, I'm gonna go and put in my equation and then get a response. Nobody does their taxes in a calculator. Nobody, you know, computes a DCF or, …”
Sivulka calls Microsoft Excel the most important software ever invented
“The spreadsheet in Excel is the most important software to have ever been invented.”
A new productivity suite will emerge for non-coding AI interaction
“We think that there's going to be an entirely new productivity suite for how humans, how experts, want to interface with AI without having to get in the weeds in code”
Hebbia was the first company to productionize RAG in 2020
“Hebbia were actually the first to turn that into a product. So it's like a very close thing to my heart. So back in 2020, we were the first people to actually productionize it, roll it out.”
Hebbia developed an undefeated re-ranker architecture that it does not use
“And we came up with a novel re-ranker architecture, which four years later, academia and industry have not beat, and we do not use it.”
The artificial intelligence base model layer will eventually become commoditized
“We believe the model layer will become commoditized.”
Hebbia processes 250 billion LLM calls per month
“I think we run around Like, two hundred and fifty billion large language model calls a month.”
Maximizer boosted Hebbia's token throughput to 500 million tokens per minute
“Before we had Maximizer, with all the rate limits that we had, we could, like, run a million tokens a minute, and now we can do 500 or four hundred fifty million tokens a minute.”
AI hallucinations are old news; modern models outperform humans
“I think it's old news. And I think the only reason we still talk about it, it's like everyone talks about hallucinations and, you know, no one knows if it's happening or what's going on. It's like fugazi fugazi. Like, it was a problem back when the models were…”
Hebbia is willing to spend $10,000 annually per user on compute
“We'll spend 10,000 dollars in model costs per user a year. We don't care.”
The San Francisco artificial intelligence ecosystem suffers from strong groupthink
“I think not a lot of people are saying it, but I think like SF has a really big group think.”
Hebbia currently employs zero artificial intelligence researchers in San Francisco
“And so we currently don't have any AI researchers hired out in San Francisco.”
The alpha is gone for starting new AI companies today
“I don't mean to be relatively pessimistic, but I wouldn't start a company in AI. I mean that from the perspective of, like, the alpha is gone.”
Enterprise AI sales are currently driven by FOMO, not traditional pain points
“Right now, it's less around pain, it's less around, like, standard enterprise SaaS cycles, and probably more around FOMO, around missed upside, around value cases that are really hard to define.”
Sivulka is "very short" Accenture, consultancies, and the Big Four
“I'm very short Accenture and consultancies, but and the big four.”
AI will transform junior finance roles without reducing total job counts
“So I firmly believe that being an investment banking junior won't look the same as it looked five years ago. I definitely believe the same for investors, for lawyers, for everyone else. But I actually don't really think that that will decrease the amount of jo…”