StarZero All shows Try the podcast clipper →

Oct 19, 2023 · 32m · no-priors

No Priors Ep. 37 | With Kawal Gandhi

Kawal Gandhi · 22m spoken Elad Gil · 3m spoken Sarah Guo · 3m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this episode of No Priors, hosts Sarah Guo and Elad Gil speak with Kawal Gandhi, Generative AI lead in the Office of the CTO at Google Cloud, about the evolution of enterprise AI infrastructure, model adoption strategies, and hardware scaling. Gandhi provides deep insights into Google's internal dogfooding, Vertex AI ecosystem, tenant data security, TPU architecture, and the transition toward multimodal applications.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. The hosts hold 24% of the talking time here. How this is scored →

The hosts as informed peer 5.1 Guest teaching 3.7 Guest disagreement 0.3 The hosts pushing back 0.4
05100:0010:0020:0030:002:23–5:23 · The hosts as informed peer 3/10 Dogfooding Internal Google AI into Workspace and Duet AI The hosts ask exploratory questions about internal Google AI dogfooding into Workspace and Duet AI. Gandhi describes product development cycles and practical features like email drafting and translation in an agreeable, collaborative tone.5:24–10:03 · The hosts as informed peer 6/10 Vertex AI, Domain-Specific Models, and Model Garden Elad demonstrates domain expertise by citing Braintrust CEO Ankur Goyal's thesis on sequential enterprise LLM adoption. Gandhi adds color from Google Cloud's Model Garden, gently distinguishing genuine builder excitement from market hype.10:03–13:21 · The hosts as informed peer 5/10 Horizontal and Vertical Enterprise Use Cases Elad proposes a three-tier framework for customer use cases (experimentation, internal tools, external features). Gandhi refines this into an organizational trust cycle progressing from efficiency to productivity to creativity.13:21–18:43 · The hosts as informed peer 4/10 The Transition from Text to Multimodal AI Sarah inquires about the progression from text to multimodality, investment costs, and operational anti-patterns. Gandhi educates on modal progressions, safety guardrails, and tenant data isolation.18:43–23:02 · The hosts as informed peer 6/10 Developer Productivity, Code Generation, and Workflow Integration Elad draws parallels between cloud customer trends and startup ecosystem patterns around developer-led adoption. Sarah drills into developer workflow integration, exploring where AI bots reside across IDEs and documentation.23:02–26:53 · The hosts as informed peer 7/10 TPU Architecture, Hardware Abstraction, and Scaling Inference Elad highlights Google's pioneering TPU development history and presses on specific hardware trade-offs against GPUs. Gandhi notes developer familiarity barriers and pivots to inference scaling architectures.26:53–31:38 · The hosts as informed peer 5/10 GPU Supply Dynamics, Multimodal Focus, and Synthetic Data Sarah questions how the Nvidia GPU shortage affects customer architectural decisions and asks about model provider demand concentration. Gandhi reframes the issue around deployment and security rather than compute supply constraints.2:23–5:23 · Guest teaching 3/10 Dogfooding Internal Google AI into Workspace and Duet AI The hosts ask exploratory questions about internal Google AI dogfooding into Workspace and Duet AI. Gandhi describes product development cycles and practical features like email drafting and translation in an agreeable, collaborative tone.5:24–10:03 · Guest teaching 4/10 Vertex AI, Domain-Specific Models, and Model Garden Elad demonstrates domain expertise by citing Braintrust CEO Ankur Goyal's thesis on sequential enterprise LLM adoption. Gandhi adds color from Google Cloud's Model Garden, gently distinguishing genuine builder excitement from market hype.10:03–13:21 · Guest teaching 4/10 Horizontal and Vertical Enterprise Use Cases Elad proposes a three-tier framework for customer use cases (experimentation, internal tools, external features). Gandhi refines this into an organizational trust cycle progressing from efficiency to productivity to creativity.13:21–18:43 · Guest teaching 4/10 The Transition from Text to Multimodal AI Sarah inquires about the progression from text to multimodality, investment costs, and operational anti-patterns. Gandhi educates on modal progressions, safety guardrails, and tenant data isolation.18:43–23:02 · Guest teaching 3/10 Developer Productivity, Code Generation, and Workflow Integration Elad draws parallels between cloud customer trends and startup ecosystem patterns around developer-led adoption. Sarah drills into developer workflow integration, exploring where AI bots reside across IDEs and documentation.23:02–26:53 · Guest teaching 4/10 TPU Architecture, Hardware Abstraction, and Scaling Inference Elad highlights Google's pioneering TPU development history and presses on specific hardware trade-offs against GPUs. Gandhi notes developer familiarity barriers and pivots to inference scaling architectures.26:53–31:38 · Guest teaching 4/10 GPU Supply Dynamics, Multimodal Focus, and Synthetic Data Sarah questions how the Nvidia GPU shortage affects customer architectural decisions and asks about model provider demand concentration. Gandhi reframes the issue around deployment and security rather than compute supply constraints.2:23–5:23 · Guest disagreement 0/10 Dogfooding Internal Google AI into Workspace and Duet AI The hosts ask exploratory questions about internal Google AI dogfooding into Workspace and Duet AI. Gandhi describes product development cycles and practical features like email drafting and translation in an agreeable, collaborative tone.5:24–10:03 · Guest disagreement 1/10 Vertex AI, Domain-Specific Models, and Model Garden Elad demonstrates domain expertise by citing Braintrust CEO Ankur Goyal's thesis on sequential enterprise LLM adoption. Gandhi adds color from Google Cloud's Model Garden, gently distinguishing genuine builder excitement from market hype.10:03–13:21 · Guest disagreement 0/10 Horizontal and Vertical Enterprise Use Cases Elad proposes a three-tier framework for customer use cases (experimentation, internal tools, external features). Gandhi refines this into an organizational trust cycle progressing from efficiency to productivity to creativity.13:21–18:43 · Guest disagreement 0/10 The Transition from Text to Multimodal AI Sarah inquires about the progression from text to multimodality, investment costs, and operational anti-patterns. Gandhi educates on modal progressions, safety guardrails, and tenant data isolation.18:43–23:02 · Guest disagreement 0/10 Developer Productivity, Code Generation, and Workflow Integration Elad draws parallels between cloud customer trends and startup ecosystem patterns around developer-led adoption. Sarah drills into developer workflow integration, exploring where AI bots reside across IDEs and documentation.23:02–26:53 · Guest disagreement 0/10 TPU Architecture, Hardware Abstraction, and Scaling Inference Elad highlights Google's pioneering TPU development history and presses on specific hardware trade-offs against GPUs. Gandhi notes developer familiarity barriers and pivots to inference scaling architectures.26:53–31:38 · Guest disagreement 1/10 GPU Supply Dynamics, Multimodal Focus, and Synthetic Data Sarah questions how the Nvidia GPU shortage affects customer architectural decisions and asks about model provider demand concentration. Gandhi reframes the issue around deployment and security rather than compute supply constraints.2:23–5:23 · The hosts pushing back 0/10 Dogfooding Internal Google AI into Workspace and Duet AI The hosts ask exploratory questions about internal Google AI dogfooding into Workspace and Duet AI. Gandhi describes product development cycles and practical features like email drafting and translation in an agreeable, collaborative tone.5:24–10:03 · The hosts pushing back 1/10 Vertex AI, Domain-Specific Models, and Model Garden Elad demonstrates domain expertise by citing Braintrust CEO Ankur Goyal's thesis on sequential enterprise LLM adoption. Gandhi adds color from Google Cloud's Model Garden, gently distinguishing genuine builder excitement from market hype.10:03–13:21 · The hosts pushing back 0/10 Horizontal and Vertical Enterprise Use Cases Elad proposes a three-tier framework for customer use cases (experimentation, internal tools, external features). Gandhi refines this into an organizational trust cycle progressing from efficiency to productivity to creativity.13:21–18:43 · The hosts pushing back 0/10 The Transition from Text to Multimodal AI Sarah inquires about the progression from text to multimodality, investment costs, and operational anti-patterns. Gandhi educates on modal progressions, safety guardrails, and tenant data isolation.18:43–23:02 · The hosts pushing back 0/10 Developer Productivity, Code Generation, and Workflow Integration Elad draws parallels between cloud customer trends and startup ecosystem patterns around developer-led adoption. Sarah drills into developer workflow integration, exploring where AI bots reside across IDEs and documentation.23:02–26:53 · The hosts pushing back 1/10 TPU Architecture, Hardware Abstraction, and Scaling Inference Elad highlights Google's pioneering TPU development history and presses on specific hardware trade-offs against GPUs. Gandhi notes developer familiarity barriers and pivots to inference scaling architectures.26:53–31:38 · The hosts pushing back 1/10 GPU Supply Dynamics, Multimodal Focus, and Synthetic Data Sarah questions how the Nvidia GPU shortage affects customer architectural decisions and asks about model provider demand concentration. Gandhi reframes the issue around deployment and security rather than compute supply constraints.

speaking balance: gold is the hosts, purple is the guest (3 minute bins)

0:00 · the hosts 28.5% · guest 71.5%0:00 · the hosts 28.5% · guest 71.5%3:00 · the hosts 20.9% · guest 79.1%3:00 · the hosts 20.9% · guest 79.1%6:00 · the hosts 21.7% · guest 78.3%6:00 · the hosts 21.7% · guest 78.3%9:00 · the hosts 24.1% · guest 75.9%9:00 · the hosts 24.1% · guest 75.9%12:00 · the hosts 12.3% · guest 87.7%12:00 · the hosts 12.3% · guest 87.7%15:00 · the hosts 19.2% · guest 80.8%15:00 · the hosts 19.2% · guest 80.8%18:00 · the hosts 40.6% · guest 59.4%18:00 · the hosts 40.6% · guest 59.4%21:00 · the hosts 28.5% · guest 71.5%21:00 · the hosts 28.5% · guest 71.5%24:00 · the hosts 22.2% · guest 77.8%24:00 · the hosts 22.2% · guest 77.8%27:00 · the hosts 18% · guest 82%27:00 · the hosts 18% · guest 82%30:00 · the hosts 29.5% · guest 70.5%30:00 · the hosts 29.5% · guest 70.5%
Sharpest disagreement ▶ 26:53 Reframing GPU shortage narrative

Gandhi softly deflects Sarah's question regarding GPU supply constraints, arguing that customer conversations shift away from chip shortages once deployment, platform security, and regionalization requirements are addressed.

 Hardest push from the hosts ▶ 24:20 Pressing on specific TPU vs GPU trade-offs

After Gandhi provides a general abstraction answer regarding hardware layers, Elad directly re-asks for concrete trade-offs and differences in developer familiarity between TPUs and GPUs.

 Biggest teaching moment ▶ 10:35 Trust framework for enterprise AI adoption

Gandhi restructures Elad's prompt on enterprise use cases by detailing how organizations move along an efficiency-to-productivity-to-trust curve before enabling autonomous agents.

 The host holds their own ▶ 7:04 Elad introduces Braintrust sequential adoption model

Elad demonstrates industry depth by citing Braintrust founder Ankur Goyal's model on how enterprises typically jump into fine-tuning prematurely before stepping back to API prototyping.
the scores for every segment, with the reasoning behind each
ChapterTopicThe hosts as informed peerGuest teachingGuest disagreementThe hosts pushing backWhy
Dogfooding Internal Google AI into Workspace and Duet AI 3300 The hosts ask exploratory questions about internal Google AI dogfooding into Workspace and Duet AI. Gandhi describes product development cycles and practical features like email drafting and translation in an agreeable, collaborative tone.
Vertex AI, Domain-Specific Models, and Model Garden 6411 Elad demonstrates domain expertise by citing Braintrust CEO Ankur Goyal's thesis on sequential enterprise LLM adoption. Gandhi adds color from Google Cloud's Model Garden, gently distinguishing genuine builder excitement from market hype.
Horizontal and Vertical Enterprise Use Cases 5400 Elad proposes a three-tier framework for customer use cases (experimentation, internal tools, external features). Gandhi refines this into an organizational trust cycle progressing from efficiency to productivity to creativity.
The Transition from Text to Multimodal AI 4400 Sarah inquires about the progression from text to multimodality, investment costs, and operational anti-patterns. Gandhi educates on modal progressions, safety guardrails, and tenant data isolation.
Developer Productivity, Code Generation, and Workflow Integration 6300 Elad draws parallels between cloud customer trends and startup ecosystem patterns around developer-led adoption. Sarah drills into developer workflow integration, exploring where AI bots reside across IDEs and documentation.
TPU Architecture, Hardware Abstraction, and Scaling Inference 7401 Elad highlights Google's pioneering TPU development history and presses on specific hardware trade-offs against GPUs. Gandhi notes developer familiarity barriers and pivots to inference scaling architectures.
GPU Supply Dynamics, Multimodal Focus, and Synthetic Data 5411 Sarah questions how the Nvidia GPU shortage affects customer architectural decisions and asks about model provider demand concentration. Gandhi reframes the issue around deployment and security rather than compute supply constraints.

Statements from this episode (12)

Prediction Not checkable as stated
Gandhi: Model drift, ops, and safety will differentiate AI platforms
“Because we've spent a lot of time really specializing around the model drift, operations, tooling, safety around it, super early, but I think those are the elements that will differentiate on the platform from the next few quarters.”
Kawal Gandhi Oct 19, 2023 ▶ 6:43
Opinion
Gandhi: Generative AI surge is genuine engineering excitement, not hype
“So I see this as early excitement. It's not hype, so I want to decouple that. It's real excitement because engineers, we love it. Like, I've been one, you want to grab something, you don't want to have restrictions around it, and you want to show the art of th…”
Kawal Gandhi Oct 19, 2023 ▶ 8:08
Insight
Gandhi: Enterprises adopt AI for efficiency before pursuing creativity
“Inside the enterprise, folks want to be creative, but every time they think about efficiency gains inside their workflows. So how can I make my workflow better so I can invest back into my group? And then how can I make productivity better than I can make them…”
Kawal Gandhi Oct 19, 2023 ▶ 10:42
Assertion Not checkable as stated
Gandhi: Enterprise AI adoption started in sales and marketing
“It started from sales and marketing. Like as soon as these came out from a horizontal perspective, I think it has an intersection between horizontal and vertical for every department, whether regulated or unregulated. There's like bottlenecks and content creat…”
Kawal Gandhi Oct 19, 2023 ▶ 12:10
Insight
Gandhi: Multimodal AI will progress sequentially from text to audio-visual integration
“Multimodal, I think we're early stages now. So you're gonna see the early models, which are audio based. So I think about it like text, then images and text with Audio. And then you're kind of combining these medias together.”
Kawal Gandhi Oct 19, 2023 ▶ 13:56
Assertion Not checkable as stated
Gandhi: Gaming is the leading industry scaling multimodal AI adoption
“I think gaming is a really good industry, which is kind of scaling out multimodality. And we have a lot of customers who are using that.”
Kawal Gandhi Oct 19, 2023 ▶ 14:40
Prediction Not checkable as stated
Gandhi: Plunging model and platform costs will drive AI adoption
“The expensive part is now becoming cheap is the models, the availability, the usage of the platform. Those were the things that were really expensive. I think you, if you do a cost craft you know, curve right now, and the investments you all are making into th…”
Kawal Gandhi Oct 19, 2023 ▶ 16:19
Assertion Not publicly verifiable
Gandhi: Google Cloud has certified over 10,000 engineers in generative AI
“We have more than 10,000 people certified now. Just more on GCP generative AI.”
Kawal Gandhi Oct 19, 2023 ▶ 17:01
Assertion Not checkable as stated
Gil: Mid-market tech companies and developers are leading AI adoption
“I'm seeing something very similar in the startup world where a lot of the early startups are basically technologists building stuff for themselves or mid-market tech companies as sort of the earliest adopters outside of a Google or a Microsoft or sort of the r…”
Elad Gil Oct 19, 2023 ▶ 20:11
Prediction Not checkable as stated
Gandhi: Future document platforms will generate product outputs directly from specs
“I really think there will be an evolution as a platform's mature where the docs can make suggestions and those suggestions could be here's based on, you know, what you've spec'd out here is output.”
Kawal Gandhi Oct 19, 2023 ▶ 21:59
Assertion Not checkable as stated
Gil: Google TPUs were dramatically more performant than GPUs for years
“And it obviously was dramatically more performant than GPU for a long time.”
Elad Gil Oct 19, 2023 ▶ 23:22
Insight
Gandhi: Enterprise AI is 60% model work and 40% legacy integration
“I think models in my mind are like, 50, 60% is the workaround, and how do you leverage your current investment is 30, 40% work that goes in from the groups, from our customers as well.”
Kawal Gandhi Oct 19, 2023 ▶ 29:34
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 100 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more