Jan 10, 2025 · 55m · latent-space
Beating Google at Search with Neural PageRank and $5M of H200s — with Will Bryk of Exa.ai
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Exa.ai CEO Will Bryk discusses rebuilding web search from first principles using neural link-prediction foundation models, explaining how variable compute, agentic workflows, and specialized retrieval infrastructure challenge legacy keyword-based search engines.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 4.5 Guest teaching 4.4 Guest disagreement 2.1 The hosts pushing back 2.5
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 48:05 Premise rejection on human search labeling
Will directly dismisses Swix's suggestion to hire grad students for human search trajectories, explaining based on Exa's internal experiments that humans default to flawed keyword matching.
Hardest push from the hosts ▶ 39:05 Pushback on Level 5 autonomous search agentsSwix uses the autonomous vehicle framework to challenge Will's vision of autonomous search, arguing that Level 5 agents fail because users lose trust when locked out of the reasoning loop.
Biggest teaching moment ▶ 21:31 Distinguishing Bing wrappers from custom neural searchWill delivers a technical breakdown showing why SearchGPT and Perplexity are effectively cached wrappers over legacy search APIs rather than scratch neural search engines.
The host holds their own ▶ 53:51 Host breakdown of search inference unit economicsSwix articulates the fundamental financial ceiling of search, contrasting Google's RPM ad revenue with LLM token costs and explaining why Exa must pre-compute embeddings during crawling rather than at inference.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Early Career Anecdotes: SpaceX, Zoox, and Autonomous Driving | 4 | 5 | 2 | 2 | Swix and Alessio probe the origin of Metaphor and ask for clarification on the link prediction foundation model. Will clarifies that the model performs document prediction rather than memorizing raw URLs, walking the hosts through the transformer-inspired training objective. | |
| Exa Architecture and the 10^18 Scaling Philosophy | 3 | 4 | 1 | 1 | Will explains the three core subsystems of Exa (crawling, neural processing index, and high-throughput vector serving). The hosts ask about the name Exa and the 10^18 philosophy versus Google's 10^100, which Will frames as filtering down to exact matches rather than returning millions of generic pages. | |
| Deep Search and Variable Compute Paradigms | 4 | 5 | 2 | 3 | Will introduces Exa's deep search launch as an o1-equivalent variable compute model for retrieval. Swix presses on how Exa guarantees completeness and manages compute credit limits across complex queries. | |
| Super Knowledge Versus Super Intelligence | 6 | 4 | 1 | 1 | Alessio and Swix share insights from venture sourcing tools and cite Karpathy's perspective on small modular intelligence units calling tools. Will agrees and articulates the theoretical difference between super intelligence and super knowledge. | |
| Neural PageRank vs. Traditional Keyword Search Engines | 5 | 6 | 3 | 3 | Swix asks how Exa differs from Perplexity and SearchGPT. Will explains that wrapper systems rely on Bing APIs with document caches, contrasting that with building an end-to-end neural search engine with neural PageRank to bypass SEO slop. | |
| Scraping Infrastructure and Navigating the Closed Web | 4 | 4 | 2 | 2 | The discussion turns to Exa's scraping API alongside competitors like Jina and Firecrawl. Swix asks how Exa navigates the increasingly closed web of paywalls and bot-blockers, with Will pointing to long-tail open data and publisher partnerships. | |
| Novel Search Verticals and How Retrieval Shapes the Web | 5 | 3 | 1 | 2 | Will lists novel search verticals including dating, academic research, and investor sourcing. Swix references McLuhanism to discuss how search algorithms shape the creator economy, which Will endorses as neural retrieval incentivizing higher quality content over keyword stuffing. | |
| LLM Interfaces, Query Intent, and Subjective Ranking | 5 | 5 | 3 | 4 | Alessio runs a live query on learning in public that fails to find Swix, prompting a debate on search intent versus subcultural keywords. Will explains why LLMs must serve as intermediate translators for human prompts and distinguishes objective filtering from subjective ranking. | |
| Agentic Search Workflows and Autonomy Trade-offs | 6 | 4 | 3 | 5 | Swix challenges full autonomy Level 5 agentic search, arguing that developers and researchers prefer drive-assist interfaces over disconnected black-box executions. Will defends the batch search paradigm while conceding that iterative previews bridge the context gap. | |
| Enterprise Search Landscape and Long-Term Horizons | 5 | 6 | 4 | 3 | After discussing o1's self-play reasoning, Swix suggests paying grad students to map human search trajectories. Will rejects the premise, explaining that human search evaluators inevitably resort to keyword matching whereas LLMs evaluate semantic relevance far more reliably. | |
| Exa Company Culture, Nap Pods, and First-Principles Building | 3 | 2 | 1 | 2 | The hosts bring up Exa's culture, including importing heavy nap pods from China and a humorous TechCrunch quote from CTO Jeff. Will discusses building from first principles with friends, closing with details on Exa's 5 million dollar H200 cluster and inference unit economics. |
Statements from this episode (21)
Bryk: Google Search hadn't changed in a decade by 2021
“And then there was Google which felt like it hadn't changed in a decade because it really hadn't. And it like, you would give it a simple query, like, I don't know shirts without stripes, and it would give you a bunch of results for the shirts with stripes.”
Bryk: SpaceX had a rule that interns could not touch Elon Musk
“I worked at SpaceX because I really just wanted to work at one of his companies, and I remember they had a rule, like, interns cannot touch Elon, and that rule actually influenced my actions.”
Bryk: Exa is the 'OpenAI of search' building AGI for retrieval
“I often say we're the OpenAI of search because we're a research company, we're a research startup that does like fundamental research into making like AGI for search in a way. And then we have all these like business products that come out of that.”
Bryk: Exa's initial foundation model predicted documents rather than raw URLs
“The link refers here to a document. It's not, I think one confusing thing is it's not, you're not actually predicting the URL itself. That would require like the system to have memorized URLs. You're actually like getting the actual document. A more accurate n…”
Exa runs core search subsystems with one or two people each
“And so it's like the crawling system, the AI processing system, and then the serving system. Those are all like, you know, teams of like hundreds, maybe thousands of people at Google. But for us, it's like one or two people each typically, but”
Bryk: Comprehensive web search requires variable compute scaled to query complexity
“What you basically have to do is you have to put more compute into the query, into the search, until you get the full comprehensiveness. And I think there's an interesting point here, which is that not all queries are made equal. Some queries, just like this b…”
Bryk: Exa applies OpenAI's o1 variable compute paradigm to web search
“One way of thinking about what we built is like O-one for search because, Oh, one is all about like, you know, some questions require more compute than others, and we'll put as much compute into the question as we need to solve it. So similarly with our search…”
Bryk: Superintelligent models like GPT-5 will fail using Google Search
“You could have a world, and we are going to have this world, where you have, like, GPT-V level systems and beyond that could, like, answer any complex request. Unless it requires some, like, if you say, like you know, give me a list of all the PhDs in New York…”
Bryk: Smaller LLMs with search tools are vastly superior to massive memorization models
“Yes, I believe this is a much more optimal system to have a smaller LLM that's really just like an intelligence module. And it makes a call to a search tool. That's way more efficient because if, okay, I mean, the opposite of that would be like the LLM is so b…”
Bryk: Google and Bing rely on limited keyword algorithms due to constraints
“Google and Bing work, and they're just not using new methods. There are all sorts of reasons for that. Like, one, like, Google has to be comprehensive over the web, so they're, and they have to return in 400 milliseconds. And those two things combined means th…”
Bryk: Perplexity and ChatGPT Search rely on legacy Google and Bing APIs
“So these systems, there are a few of them now they basically rely on like traditional search engines like Google or Bing, and then they combine them with like LLMs at the end to, you know, output some power graphics answering your question. So they, Like, Sear…”
Bryk: Neural link prediction is strictly more powerful than Google's PageRank
“The link prediction objective can be seen as like a neural page rank, because what you're doing is you're predicting the links people share. And so if everyone is sharing some Paul Graham essay about fundraising, then like our model is more likely to predict i…”
Bryk: Searching the open long tail captures most web search value
“One response is just that there's so much value in the long tail of sites that are open. And just like, even just searching over those well gets you most of the value.”
Bryk: Content creators will establish data partnerships with search entities
“I do see the world as like the future where the data The data producers, the content creators will make partnerships with the entities that find that data.”
Bryk: The internet's content strictly mirrors what search engines optimize for
“Whatever the search engine optimizes for is what the internet looks like.”
Bryk: Google and Bing completely fail when given paragraph queries
“Traditional search engines like Google or Bing, they're actually designed for humans typing keywords. If you give a paragraph to Google or Bing, they just completely fail.”
Exa.ai uses LLMs as data labelers instead of humans
“We could get by, which we are right now doing, using, like, LLMs as the labelers.”
Bryk: Search engines do not need PhD-level training data
“With search, you're asking, like, simple questions about billions of things, like, is this a startup? Did this person write a blog post about search? You know, those are actually simple questions. You don't need, like, PhD level training data.”
Bryk: Exa purchased a $5 million Nvidia H200 GPU cluster
“We now have we just purchased a five million dollar H 200 cluster.”
Bryk: 200x drop in LLM costs requires rethinking search from scratch
“When some very useful tool goes down in cost by 200 X in like the space of, I don't know, a couple years, There are going to be new opportunities in search, right? So like, to not integrate this and build up, to not like rethink search from scratch, the search…”