Apr 25, 2023 · 35m · no-priors
No Priors Ep. 8 | With Neeva’s Sridhar Ramaswamy
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of No Priors, Neeva co-founder and former Google executive Sridhar Ramaswamy explores the reinvention of web search through AI-generated answers, the economics of LLM inference, and the future of autonomous agents.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 15.3% of the talking time here. How this is scored →
The hosts as informed peer 4.0 Guest teaching 3.3 Guest disagreement 1.6 The hosts pushing back 1.1
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 27:27 Dismissing tech billionaire defenses of advertising
Sridhar bluntly rejects the notion that advertising serves purely altruistic or quality-filtering purposes, dismissing billionaire explanations as convenient, self-serving corporate narratives.
Hardest push from the hosts ▶ 26:58 Elad pressing on willingness-to-pay as a relevance signalElad pushes back on the purely negative framing of search ads by raising the historic Google argument that commercial bid value might indicate link quality.
Biggest teaching moment ▶ 14:47 Breaking down unit economics of LLM inference vs RPMsSridhar walks through the exact math of 5-cent model calls yielding $50 CPMs versus average US search RPMs, demonstrating why massive foundation models must be replaced by fine-tuned smaller models for search.
The host holds their own ▶ 9:33 Elad citing early Google mobile search and one-box mechanicsElad demonstrates domain expertise by detailing how Google previously solved instant answers through custom-indexed one-boxes and specialized ranking pipelines.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| The Founding Motivation and Evolution of Neeva | 3 | 2 | 1 | 1 | Sarah sets a welcoming tone, asking Sridhar about the founding thesis of Neeva and the friction of consumer habits around default search engines. Sridhar reflects collaboratively on lessons in consumer psychology and regional differences between the US and Europe. | |
| The Shift to AI Summaries and Direct Answers | 2 | 4 | 1 | 0 | Sarah asks about user reception of AI summaries. Sridhar delivers an extensive technical and historical explanation of Google's featured snippets, the failure of older NLP models to scale, and why LLM-driven cited summaries unlock direct answers for a majority of queries. | |
| Retrieval-Augmented Generation and Search Limits | 6 | 3 | 2 | 2 | Elad draws directly on his past Google experience building mobile search and specialized 'one-box' indices to probe the limits and coverage of LLMs for IR. Sridhar explains how retrieval-augmented generation and smaller specialized models replace fragile legacy parsing code. | |
| The Economics and Serving Costs of LLM Search | 6 | 4 | 2 | 2 | Elad brings up unit economics, Satya Nadella's competitive posture, and query-level serving costs. Sridhar breaks down query RPMs versus CPM costs, arguing that fine-tuned 5B to 10B parameter models avoid astronomical inference bills. | |
| Engineering an Independent Search Engine at Scale | 4 | 4 | 1 | 1 | Sarah inquires about the practical engineering hurdles of crawling and indexing independent search architecture on a startup budget. Sridhar explains flash-based index architectures and using LLMs as shortcuts for query rewriting. | |
| Overcoming the Distribution Challenge in Search | 5 | 5 | 5 | 3 | Elad plays devil's advocate regarding whether ad bids serve as genuine relevance signals. Sridhar rejects this framing forcefully, calling executive justifications for ads self-serving rationalizations and criticizing Amazon's search experience. | |
| AI Search and the Future of Content Publishers | 4 | 4 | 1 | 1 | Sarah asks how direct answers will affect the economic incentives of web publishers. Sridhar predicts content centralization, where major platforms wall off their data while smaller blogs face severe monetization disruption. | |
| Emerging AI Frontiers: Agents and Action Transformers | 4 | 3 | 1 | 0 | Elad asks about broader AI disruptions beyond search. Sridhar outlines potential shifts in personalized advertising and details his excitement for action transformers and agents that interface with developer and IT workflows. | |
| The Democratization of AI and Concluding Reflections | 2 | 1 | 0 | 0 | The hosts and guest conclude with an optimistic reflection on the democratization of AI platforms, comparing the current development wave to the early mobile platform explosion. |
Statements from this episode (21)
Ramaswamy: Neeva's First Three Years Taught Harsh Lessons on Consumer Readiness
“So the first three years of Neva were really about building a better private search engine. And honestly, it also taught us a lot of pretty harsh lessons about consumers and you know, whether they were ready for change or not.”
Ramaswamy: European Consumers Care More About Privacy, Aiding Neeva
“For us, for example, we were surprised that we did so much better in Europe compared to the United States, and you don't really think of them as being that different, but in practice, in terms of how many people care, it is actually very different.”
Ramaswamy: Google Beat Live.com Image Search Through Main Results Integration
“Google knocked out live.com Bing's image search as the top image search product in the world by integrating image search right into the search experience.”
Ramaswamy: Google Featured Snippets Never Passed 7 Percent Coverage
“Things like featured snippets were never deployable at scale. The technology simply was not there. Even if Google put the full might of its mighty machine against the problem, the coverage never really extended beyond like five, six, seven percent, and it woul…”
Ramaswamy: Neeva Built In-House Summarization Over OpenAI for Scale
“We decided that we didn't really want to be beholden to say like using OpenAI's API. For doing things like summarizing a four billion page index. We built a lot of the technology in-house”
Ramaswamy: Users Always Prefer Direct Answers Because They Dislike Clicks
“If you can provide a believable answer to a question, people are always going to prefer that over any number of links that you can give them. People don't like clicking on links.”
Ramaswamy: Small Language Models Generalize HTML Parsing and Scraping
“All of us have nightmares about writing beautiful soup code in order to parse web pages. It's basically regular expression parsing over ever-changing websites. It's horrible. We have done a bunch of it in the first two-ish years of Neva. That stuff is also eas…”
Ramaswamy: Usefulness Strongly Limits Answerable Questions in AI Search
“At this point, I don't feel that there's like a natural limit to how much LLMs can be used with search. I do feel, however, that there's a very strong limit to how many questions can be usefully answered.”
Ramaswamy: Search RPM Is Roughly $50 in US, $20 Globally
“The average RPM for US queries is about
40 to 50 dollars, and clearly that will be a very high cost.
the rest of the world is a lot lower, by the way, like my memory is on the order of 20 dollars if you average over the whole world.”
Ramaswamy: Model Size and Serving Costs Will Drop for Search
“For a lot of like known problems model size is not really going to be an issue and there's going to be an ongoing reduction both in the size and therefore the cost to serve them.”
Ramaswamy: Google Gave Over 100 Percent Rev Share to Early Partners
“Google gave more than a hundred percent to AOL and close to a hundred percent to Yahoo in its early years.”
Ramaswamy: Flash Let Neeva Solve Index Replacement Faster Than Google
“Asim, for example was just brilliant at engineering a system that ran completely on Flash in which we could do things like super rapid iteration, replace the entire index or the space of two days or put in arbitrary amounts of information for experimentation i…”
Ramaswamy: LLMs Can Replace Massive Click Data for Core Search Tasks
“We realized that a lot of problems that Google solved with massive scale and user data, they could, in fact, solve with LLMs. So we use a lot of them for things like query rewriting.”
Ramaswamy: Ad-Supported Platforms Carry Self-Destructive Incentives to Increase Density
“Ads sort of come you know, with elements of self-destruction built in. It's part for the course when you're doing it, it's always attractive to do things like show more ads.”
Ramaswamy: Amazon Search Is a Joke Polluted by Misleading Ads
“I find Amazon search experience a joke because it is so full of ads and it's actually misleading ads where it's really hard to find what is going on.”
Ramaswamy: Monetizing Answer-Focused AI Interfaces with Ads Is Much Harder
“If you're in the business of providing answers like ChatGPT was ads just becomes a whole lot harder to do. They're really, you're just you know, you're betting on the quality of answers”
Ramaswamy: Big Tech Claims That Ads Democratize Access Are Entirely Self-Serving
“I find this whole thing of ads enable Google to make free products or ads enable Facebook to be available for Ecuadorian people made by billionaires sitting in Palo Alto to be entirely self-serving.”
Ramaswamy Predicts Major Platforms Will Allow Indexing but Block LLM Scrapers
“I think what is going to happen is that some of the larger content creators, you know, I would put people like Reddit and Quora. These are some of the forward thinking ones very much in that bucket. They're going to say, we want to be part of search, but we do…”
Ramaswamy: AI Answers Will Consolidate Content Creation, Hurting Small Blogs
“It does feel like there might be more centralization or more consolidation when it comes to content creation. Your average small blog, which could subsidize itself or which could monetize itself with advertising is going to find it hard to compete in this answ…”
Ramaswamy: Multimodal AI-Generated Advertising Will Be a Major Disruption
“Obvious places where content is generated actually, ironically, is going to be advertising. I can see how personalized advertising actually plays a pretty big role, especially when it gets to be multimodal.”
Ramaswamy: Action Transformers Will Be Powerful but Remain Extremely Nascent
“I think, like, action transformers is going to be an incredibly powerful area. The technology is very nascent, so unlike, say, you know, OpenAI's ability to crank out new generations of LLMs, I don't think that tech is yet at a point where people can build lot…”