Oct 5, 2023 · 23m · no-priors

No Priors Ep. 35 | With Sarah Guo and Elad Gil

Elad Gil · 11m spoken Sarah Guo · 10m spoken

0:00 / 0:00

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this episode of No Priors, Sarah Guo and Elad Gil examine the architectural innovations—such as fine-tuning, RAG, and RLAIF—enabling 10x performance gains in AI, while analyzing Meta's open-source ecosystem strategy, emerging consumer social applications, and tactical playbooks for startup founders.

Chapters · click to jump

2 statements ▾

Gil: Existing models yield 10x to 100x gains without base model scaling open →

Gil: Magic Is Building Long Context AI to Ingest Entire Code Repositories open →

1 statement ▾

Guo: Trustworthiness, Cost, and Freshness Drive Adoption of RAG Over Retraining open →

2 statements ▾

Gil: Google paper shows AI feedback matches human feedback in specific cases open →

Gil: Google's Med-PaLM 2 Output Was More Accurate Than Physician Experts open →

2 statements ▾

Guo: Meta's open-source models will remain a sustained alternative to proprietary labs open →

Gil: IBM sponsored Linux with up to $1B annually in late 1990s open →

3 statements ▾

Gil: TikTok was the last major social network launched open →

Gil: Big social incumbents may benefit most from generative AI open →

Gil: AI entrepreneurs are ignoring consumer social for enterprise open →

1 statement ▾

Gil: Early in platform shifts, founders should target easy low-hanging fruit open →

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 100% of the talking time here. How this is scored →

The hosts as informed peer 6.5 Guest teaching 1.8 Guest disagreement 0.3 The hosts pushing back 0.5

speaking balance: gold is the hosts, purple is the guest (3 minute bins)

Sharpest disagreement ▶ 14:02 Defending Meta open source strategy

Sarah counters Elad's skepticism regarding open-source sustainability by highlighting how Meta offsets development costs and avoids compute/vendor lock-in.

Hardest push from the hosts ▶ 13:19 Challenging the MySQL open-source analogy

Elad directly challenges Sarah's historical framing, invoking IBM's billion-dollar subsidies of Linux against Microsoft to question whether Meta's open source move is truly analogous to MySQL.

Biggest teaching moment ▶ 18:31 Toutiao algorithmic cold start analysis

Sarah explains in granular technical detail how Toutiao bootstrapped implicit user preference models rather than relying on explicit onboarding selections.

The host holds their own ▶ 0:42 Framework for non-scaling model breakthroughs

Elad establishes the episode's intellectual backbone by enumerating the architectural innovations available on existing models without waiting for next-gen pre-training scaling.

the scores for every segment, with the reasoning behind each

Topic	The hosts as informed peer	Guest teaching	Guest disagreement	The hosts pushing back	Why
Six Pillars for 10x and 100x AI Improvements	7	2	0	0	Elad categorizes the technological levers for 10x-100x AI improvements into six distinct architectural pillars, from multimodality to context routing. The dynamic is fully collaborative and explanatory with no friction.
The Evolution and Enterprise Value of Fine-Tuning	6	2	0	0	Sarah and Elad examine the role of fine-tuning versus pre-training general models, referencing ChatGPT's launch through RLHF and enterprise proprietary datasets. Both hosts demonstrate deep familiarity with model tuning economics.
Retrieval-Augmented Generation and Hallucination Mitigation	6	1	0	0	Sarah outlines RAG's strengths in freshness, cost, and citation control, while Elad builds on the premise by detailing how retrieval systems mitigate hallucination risks.
Scaling Optimization via Reinforcement Learning from AI Feedback	7	3	2	3	After discussing RLAIF and Google's Med-PaLM 2, the discussion shifts to Meta's open source strategy. Elad pushes back against Sarah's MySQL analogy by citing IBM's multi-billion dollar sponsorship of Linux as a counterweight to Microsoft.
Generative AI as the Catalyst for Next-Gen Consumer Social	7	2	0	0	Elad analyzes the stagnation of social network mechanics since TikTok and maps social products across axes of broadcast vs mutual and modality. Sarah complements this by breaking down Toutiao's algorithmic cold-start model.
Founder Strategy: Navigating Early-Stage AI Market Selection	6	1	0	0	Elad offers strategic guidance for early-stage AI founders to target low-hanging fruit over complex multi-year research bets. Sarah strongly reinforces this perspective with observations from portfolio accelerator founders.

Statements from this episode (11)

Insight

Gil: Existing models yield 10x to 100x gains without base model scaling

“In order to 10 X or even a hundred X use cases and usages for AI outside of that, there's things that could just be done on existing models today. So you don't need to wait for GPT seven or whatever. You could start with GPT four or GPT 3.5 and add these thing…”

Elad Gil Oct 5, 2023 ▶ 0:49

Assertion Supported

Gil: Magic Is Building Long Context AI to Ingest Entire Code Repositories

“Magic, for example, is doing that for code. You know, you should be able to dump an entire code repo into a coding model instead of having to do it piecemeal.”

Elad Gil Oct 5, 2023 ▶ 1:35

Insight

Guo: Trustworthiness, Cost, and Freshness Drive Adoption of RAG Over Retraining

“I think of the core driver as like trustworthiness, right? Citation control of information source. And so now you have this architecture where people are using think of it as like traditional information retrieval techniques and search in combination with thes…”

Sarah Guo Oct 5, 2023 ▶ 7:54

Assertion Supported

Gil: Google paper shows AI feedback matches human feedback in specific cases

“Google just came out with a really interesting paper on that, where, you know, they showed that you can have an AI similarly provide feedback to whether the AI itself is generating good output. And for certain use cases, that works as well as people.”

Elad Gil Oct 5, 2023 ▶ 9:47

Assertion Supported

Gil: Google's Med-PaLM 2 Output Was More Accurate Than Physician Experts

“They trained a model specifically on medical data, and the output from the model tended to be more correct than human physician experts.”

Elad Gil Oct 5, 2023 ▶ 10:15

Prediction Not checkable as stated

Guo: Meta's open-source models will remain a sustained alternative to proprietary labs

“And I think like, I think it is very likely to be a big mover in the ecosystem because if they sponsor some Baseline of models that are big enough to be valuable, high quality enough to be valuable with Facebook AI research. And then enough people find these m…”

Sarah Guo Oct 5, 2023 ▶ 12:45

Assertion Supported

Gil: IBM sponsored Linux with up to $1B annually in late 1990s

“And Linux in part was very much sponsored by IBM throughout the late nineties to the tune of in some years, a billion dollars a year.”

Elad Gil Oct 5, 2023 ▶ 13:25

Assertion Not checkable as stated

Gil: TikTok was the last major social network launched

“I think more broadly in social and AI, it's kind of striking that the last large social network in some sense was TikTok, which was launched seven years ago now.”

Elad Gil Oct 5, 2023 ▶ 16:21

Prediction Not checkable as stated

Gil: Big social incumbents may benefit most from generative AI

“And I think the big social networks like Meta and Twitter and others may actually be the biggest beneficiaries of this new wave, but there also should be room for startups.”

Elad Gil Oct 5, 2023 ▶ 17:34

Opinion

Gil: AI entrepreneurs are ignoring consumer social for enterprise

“It's kind of this oddly Almost ignored an area from a entrepreneurship and founder perspective right now. Everybody's rushing at the enterprise stuff and the infrastructure and, you know, that whole stack.”

Elad Gil Oct 5, 2023 ▶ 20:50

Insight

Gil: Early in platform shifts, founders should target easy low-hanging fruit

“Early in markets, like when a new technology shifts and disrupts the whole market, you actually want to just do the easy stuff, right? Why do the hard stuff? There's so much low hanging fruit. Why don't you just go after the stuff? It's super easy. And my sort…”

Elad Gil Oct 5, 2023 ▶ 21:32

No Priors Ep. 35 | With Sarah Guo and Elad Gil

How this conversation actually went

Statements from this episode (11)

Turn any episode into a week of clips.