StarZero All shows Try the podcast clipper →

Sep 20, 2023 · 50m · mad

Beyond ChatGPT: Ori Goshen’s Playbook for Building Neuro-Symbolic LLMs

Ori Goshen · 37m spoken Matt Turck · 7m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this episode of The MAD Podcast, AI21 Labs Co-CEO Ori Goshen discusses how neuro-symbolic AI, modular system orchestration, and cloud-neutral enterprise partnerships are paving the way for reliable, production-ready AI beyond standard LLM scaling.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 16.9% of the talking time here. How this is scored →

Matt as informed peer 3.3 Guest teaching 3.3 Guest disagreement 0.6 Matt pushing back 1.1
05100:0015:0030:0045:000:13–6:13 · Matt as informed peer 4/10 Series C Funding and Strategic AI Fundraising Matt demonstrates knowledge of AI capital dynamics and GPU costs, challenging Ori on whether raising the most money determines the winner. Ori gently reframes the premise, arguing that raising excess capital isn't healthy and emphasizing efficient compute deployment.6:13–10:42 · Matt as informed peer 3/10 Founding AI21 Labs and Neuro-Symbolic AI Vision Matt highlights the significance of 2017 for NLP and recognizes co-founder Amnon Shashua from Mobileye. Ori educates Matt on the historical shift from 1980s expert symbolic systems to deep learning and why neuro-symbolic AI is required for deterministic reliability.10:42–20:11 · Matt as informed peer 7/10 Deconstructing LLM Weaknesses and AI System Orchestration Matt demonstrates notable domain depth by bringing up the Yann LeCun vs. Gary Marcus debate and Josh Tenenbaum's research at MIT, questioning whether pure deep learning still outperforms hybrid neuro-symbolic approaches. Ori defends his thesis by highlighting diminishing returns on model scaling and practical compute economics.20:11–25:51 · Matt as informed peer 3/10 Jurassic-2 Models and Multi-Cloud Strategy Matt asks how developers decide between Jurassic-2 Light, Mid, and Ultra models. Ori details trade-offs in context length, latency, cost, and multi-cloud deployment options.25:51–33:47 · Matt as informed peer 3/10 Enterprise Task-Specific Systems and Contextual Answers Ori explains why 95% of enterprise use cases require narrow task-specific modular systems rather than general-purpose chat LLMs. Matt seeks clarification on where Contextual Answers fits into AI21's overall product taxonomy.33:47–35:57 · Matt as informed peer 2/10 Wordtune Application and AI User Experience Ori details Wordtune's user growth and monetization model, explaining why AI21 intentionally avoided a chat interface in favor of writing extensions. Matt listens and moves the topic forward.35:57–43:06 · Matt as informed peer 3/10 Enterprise AI Adoption and Production Challenges Matt references Ori's earlier remark about moving from exploration to experimentation, prompting a discussion on enterprise production timelines. Ori outlines the technical and financial hurdles enterprise customers face over 6 to 12 month horizons.43:06–45:41 · Matt as informed peer 3/10 Go-To-Market and Partner Ecosystem Strategy Matt asks about go-to-market execution given AI21's broad surface area across foundation models, task systems, and consumer apps. Ori outlines their top-down high-touch enterprise sales motion alongside cloud hyperscaler partnerships.45:41–50:33 · Matt as informed peer 2/10 The Israeli AI Ecosystem and Talent Landscape Matt invites Ori to provide an overview of the Israeli AI talent ecosystem. Ori details the historical evolution from hardware and cyber into AI, noting how top academic researchers returning from US PhD programs boosted local talent.0:13–6:13 · Guest teaching 3/10 Series C Funding and Strategic AI Fundraising Matt demonstrates knowledge of AI capital dynamics and GPU costs, challenging Ori on whether raising the most money determines the winner. Ori gently reframes the premise, arguing that raising excess capital isn't healthy and emphasizing efficient compute deployment.6:13–10:42 · Guest teaching 4/10 Founding AI21 Labs and Neuro-Symbolic AI Vision Matt highlights the significance of 2017 for NLP and recognizes co-founder Amnon Shashua from Mobileye. Ori educates Matt on the historical shift from 1980s expert symbolic systems to deep learning and why neuro-symbolic AI is required for deterministic reliability.10:42–20:11 · Guest teaching 5/10 Deconstructing LLM Weaknesses and AI System Orchestration Matt demonstrates notable domain depth by bringing up the Yann LeCun vs. Gary Marcus debate and Josh Tenenbaum's research at MIT, questioning whether pure deep learning still outperforms hybrid neuro-symbolic approaches. Ori defends his thesis by highlighting diminishing returns on model scaling and practical compute economics.20:11–25:51 · Guest teaching 3/10 Jurassic-2 Models and Multi-Cloud Strategy Matt asks how developers decide between Jurassic-2 Light, Mid, and Ultra models. Ori details trade-offs in context length, latency, cost, and multi-cloud deployment options.25:51–33:47 · Guest teaching 4/10 Enterprise Task-Specific Systems and Contextual Answers Ori explains why 95% of enterprise use cases require narrow task-specific modular systems rather than general-purpose chat LLMs. Matt seeks clarification on where Contextual Answers fits into AI21's overall product taxonomy.33:47–35:57 · Guest teaching 2/10 Wordtune Application and AI User Experience Ori details Wordtune's user growth and monetization model, explaining why AI21 intentionally avoided a chat interface in favor of writing extensions. Matt listens and moves the topic forward.35:57–43:06 · Guest teaching 3/10 Enterprise AI Adoption and Production Challenges Matt references Ori's earlier remark about moving from exploration to experimentation, prompting a discussion on enterprise production timelines. Ori outlines the technical and financial hurdles enterprise customers face over 6 to 12 month horizons.43:06–45:41 · Guest teaching 2/10 Go-To-Market and Partner Ecosystem Strategy Matt asks about go-to-market execution given AI21's broad surface area across foundation models, task systems, and consumer apps. Ori outlines their top-down high-touch enterprise sales motion alongside cloud hyperscaler partnerships.45:41–50:33 · Guest teaching 4/10 The Israeli AI Ecosystem and Talent Landscape Matt invites Ori to provide an overview of the Israeli AI talent ecosystem. Ori details the historical evolution from hardware and cyber into AI, noting how top academic researchers returning from US PhD programs boosted local talent.0:13–6:13 · Guest disagreement 2/10 Series C Funding and Strategic AI Fundraising Matt demonstrates knowledge of AI capital dynamics and GPU costs, challenging Ori on whether raising the most money determines the winner. Ori gently reframes the premise, arguing that raising excess capital isn't healthy and emphasizing efficient compute deployment.6:13–10:42 · Guest disagreement 0/10 Founding AI21 Labs and Neuro-Symbolic AI Vision Matt highlights the significance of 2017 for NLP and recognizes co-founder Amnon Shashua from Mobileye. Ori educates Matt on the historical shift from 1980s expert symbolic systems to deep learning and why neuro-symbolic AI is required for deterministic reliability.10:42–20:11 · Guest disagreement 2/10 Deconstructing LLM Weaknesses and AI System Orchestration Matt demonstrates notable domain depth by bringing up the Yann LeCun vs. Gary Marcus debate and Josh Tenenbaum's research at MIT, questioning whether pure deep learning still outperforms hybrid neuro-symbolic approaches. Ori defends his thesis by highlighting diminishing returns on model scaling and practical compute economics.20:11–25:51 · Guest disagreement 0/10 Jurassic-2 Models and Multi-Cloud Strategy Matt asks how developers decide between Jurassic-2 Light, Mid, and Ultra models. Ori details trade-offs in context length, latency, cost, and multi-cloud deployment options.25:51–33:47 · Guest disagreement 1/10 Enterprise Task-Specific Systems and Contextual Answers Ori explains why 95% of enterprise use cases require narrow task-specific modular systems rather than general-purpose chat LLMs. Matt seeks clarification on where Contextual Answers fits into AI21's overall product taxonomy.33:47–35:57 · Guest disagreement 0/10 Wordtune Application and AI User Experience Ori details Wordtune's user growth and monetization model, explaining why AI21 intentionally avoided a chat interface in favor of writing extensions. Matt listens and moves the topic forward.35:57–43:06 · Guest disagreement 0/10 Enterprise AI Adoption and Production Challenges Matt references Ori's earlier remark about moving from exploration to experimentation, prompting a discussion on enterprise production timelines. Ori outlines the technical and financial hurdles enterprise customers face over 6 to 12 month horizons.43:06–45:41 · Guest disagreement 0/10 Go-To-Market and Partner Ecosystem Strategy Matt asks about go-to-market execution given AI21's broad surface area across foundation models, task systems, and consumer apps. Ori outlines their top-down high-touch enterprise sales motion alongside cloud hyperscaler partnerships.45:41–50:33 · Guest disagreement 0/10 The Israeli AI Ecosystem and Talent Landscape Matt invites Ori to provide an overview of the Israeli AI talent ecosystem. Ori details the historical evolution from hardware and cyber into AI, noting how top academic researchers returning from US PhD programs boosted local talent.0:13–6:13 · Matt pushing back 2/10 Series C Funding and Strategic AI Fundraising Matt demonstrates knowledge of AI capital dynamics and GPU costs, challenging Ori on whether raising the most money determines the winner. Ori gently reframes the premise, arguing that raising excess capital isn't healthy and emphasizing efficient compute deployment.6:13–10:42 · Matt pushing back 0/10 Founding AI21 Labs and Neuro-Symbolic AI Vision Matt highlights the significance of 2017 for NLP and recognizes co-founder Amnon Shashua from Mobileye. Ori educates Matt on the historical shift from 1980s expert symbolic systems to deep learning and why neuro-symbolic AI is required for deterministic reliability.10:42–20:11 · Matt pushing back 5/10 Deconstructing LLM Weaknesses and AI System Orchestration Matt demonstrates notable domain depth by bringing up the Yann LeCun vs. Gary Marcus debate and Josh Tenenbaum's research at MIT, questioning whether pure deep learning still outperforms hybrid neuro-symbolic approaches. Ori defends his thesis by highlighting diminishing returns on model scaling and practical compute economics.20:11–25:51 · Matt pushing back 1/10 Jurassic-2 Models and Multi-Cloud Strategy Matt asks how developers decide between Jurassic-2 Light, Mid, and Ultra models. Ori details trade-offs in context length, latency, cost, and multi-cloud deployment options.25:51–33:47 · Matt pushing back 1/10 Enterprise Task-Specific Systems and Contextual Answers Ori explains why 95% of enterprise use cases require narrow task-specific modular systems rather than general-purpose chat LLMs. Matt seeks clarification on where Contextual Answers fits into AI21's overall product taxonomy.33:47–35:57 · Matt pushing back 0/10 Wordtune Application and AI User Experience Ori details Wordtune's user growth and monetization model, explaining why AI21 intentionally avoided a chat interface in favor of writing extensions. Matt listens and moves the topic forward.35:57–43:06 · Matt pushing back 0/10 Enterprise AI Adoption and Production Challenges Matt references Ori's earlier remark about moving from exploration to experimentation, prompting a discussion on enterprise production timelines. Ori outlines the technical and financial hurdles enterprise customers face over 6 to 12 month horizons.43:06–45:41 · Matt pushing back 1/10 Go-To-Market and Partner Ecosystem Strategy Matt asks about go-to-market execution given AI21's broad surface area across foundation models, task systems, and consumer apps. Ori outlines their top-down high-touch enterprise sales motion alongside cloud hyperscaler partnerships.45:41–50:33 · Matt pushing back 0/10 The Israeli AI Ecosystem and Talent Landscape Matt invites Ori to provide an overview of the Israeli AI talent ecosystem. Ori details the historical evolution from hardware and cyber into AI, noting how top academic researchers returning from US PhD programs boosted local talent.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 28.4% · guest 71.6%0:00 · Matt 28.4% · guest 71.6%3:00 · Matt 1.6% · guest 98.4%3:00 · Matt 1.6% · guest 98.4%6:00 · Matt 17.5% · guest 82.5%6:00 · Matt 17.5% · guest 82.5%9:00 · Matt 14.6% · guest 85.4%9:00 · Matt 14.6% · guest 85.4%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 36% · guest 64%15:00 · Matt 36% · guest 64%18:00 · Matt 33% · guest 67%18:00 · Matt 33% · guest 67%21:00 · Matt 16.7% · guest 83.3%21:00 · Matt 16.7% · guest 83.3%24:00 · Matt 19.8% · guest 80.2%24:00 · Matt 19.8% · guest 80.2%27:00 · Matt 0% · guest 100%27:00 · Matt 0% · guest 100%30:00 · Matt 6.8% · guest 93.2%30:00 · Matt 6.8% · guest 93.2%33:00 · Matt 5.5% · guest 94.5%33:00 · Matt 5.5% · guest 94.5%36:00 · Matt 15.8% · guest 84.2%36:00 · Matt 15.8% · guest 84.2%39:00 · Matt 18.6% · guest 81.4%39:00 · Matt 18.6% · guest 81.4%42:00 · Matt 26.1% · guest 73.9%42:00 · Matt 26.1% · guest 73.9%45:00 · Matt 34.4% · guest 65.6%45:00 · Matt 34.4% · guest 65.6%48:00 · Matt 13.8% · guest 86.2%48:00 · Matt 13.8% · guest 86.2%
Sharpest disagreement ▶ 3:04 Guest rejects capital hyper-inflation framing

Ori directly challenges Matt's premise that raising the most capital wins the AI race, explicitly stating that accumulating massive capital isn't a healthy mindset and emphasizing efficient deployment over sheer round size.

 Hardest push from Matt ▶ 15:31 Host challenges neuro-symbolic premise with academic arguments

Matt pushes back on AI21's core thesis by bringing up the LeCun vs. Marcus debate and citing MIT research indicating pure deep learning models frequently outperform hybrid neuro-symbolic systems.

 Biggest teaching moment ▶ 26:23 Guest educates on why general LLMs fail in production

Ori breaks down why standard LLMs fall short in enterprise settings, teaching how task-specific wrapper systems with input verification outperform monolithic LLMs in reliability and cost efficiency.

 Matt holds his own ▶ 15:31 Host drops AI academic knowledge and counter-evidence

Matt showcases deep industry knowledge by citing specific academic figures (Yann LeCun, Gary Marcus, Josh Tenenbaum) and early empirical results to challenge the guest's foundational technical strategy.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Series C Funding and Strategic AI Fundraising 4322 Matt demonstrates knowledge of AI capital dynamics and GPU costs, challenging Ori on whether raising the most money determines the winner. Ori gently reframes the premise, arguing that raising excess capital isn't healthy and emphasizing efficient compute deployment.
Founding AI21 Labs and Neuro-Symbolic AI Vision 3400 Matt highlights the significance of 2017 for NLP and recognizes co-founder Amnon Shashua from Mobileye. Ori educates Matt on the historical shift from 1980s expert symbolic systems to deep learning and why neuro-symbolic AI is required for deterministic reliability.
Deconstructing LLM Weaknesses and AI System Orchestration 7525 Matt demonstrates notable domain depth by bringing up the Yann LeCun vs. Gary Marcus debate and Josh Tenenbaum's research at MIT, questioning whether pure deep learning still outperforms hybrid neuro-symbolic approaches. Ori defends his thesis by highlighting diminishing returns on model scaling and practical compute economics.
Jurassic-2 Models and Multi-Cloud Strategy 3301 Matt asks how developers decide between Jurassic-2 Light, Mid, and Ultra models. Ori details trade-offs in context length, latency, cost, and multi-cloud deployment options.
Enterprise Task-Specific Systems and Contextual Answers 3411 Ori explains why 95% of enterprise use cases require narrow task-specific modular systems rather than general-purpose chat LLMs. Matt seeks clarification on where Contextual Answers fits into AI21's overall product taxonomy.
Wordtune Application and AI User Experience 2200 Ori details Wordtune's user growth and monetization model, explaining why AI21 intentionally avoided a chat interface in favor of writing extensions. Matt listens and moves the topic forward.
Enterprise AI Adoption and Production Challenges 3300 Matt references Ori's earlier remark about moving from exploration to experimentation, prompting a discussion on enterprise production timelines. Ori outlines the technical and financial hurdles enterprise customers face over 6 to 12 month horizons.
Go-To-Market and Partner Ecosystem Strategy 3201 Matt asks about go-to-market execution given AI21's broad surface area across foundation models, task systems, and consumer apps. Ori outlines their top-down high-touch enterprise sales motion alongside cloud hyperscaler partnerships.
The Israeli AI Ecosystem and Talent Landscape 2400 Matt invites Ori to provide an overview of the Israeli AI talent ecosystem. Ori details the historical evolution from hardware and cyber into AI, noting how top academic researchers returning from US PhD programs boosted local talent.

Statements from this episode (10)

Insight
Goshen: Deep learning is necessary but not sufficient for reliable AI
“Deep learning is, is amazing. And with that said, it's necessary, but not sufficient component.”
Ori Goshen Sep 20, 2023 ▶ 8:52
Prediction Not checkable as stated
Goshen: The future of artificial intelligence will be neuro-symbolic
“The future is going to be neuro symbolic.”
Ori Goshen Sep 20, 2023 ▶ 10:10
Prediction Not checkable as stated
Goshen: In two years, focus will shift from LLMs to multi-model AI systems
“In two years from now, I guess we won't be excited and we won't be speaking about large language models. We'll be speaking about AI systems that use maybe several language models and they orchestrate this solution problem solution in such a way that is more re…”
Ori Goshen Sep 20, 2023 ▶ 14:45
Assertion Not checkable as stated
Goshen: Scaling LLM size is already showing diminishing returns
“And we're already start seeing diminishing returns. I mean, you take the architecture, you increase the size of the models or, and you see that At these levels of scale where, you know, we are dealing with things, diminishing returns, we start seeing it.”
Ori Goshen Sep 20, 2023 ▶ 17:38
Insight
Goshen: Enterprise customers heavily prioritize LLM portability across multi-cloud environments
“I was surprised how much customers care about having the portability and capability of working with a single vendor that can run under different cloud, but not only cloud, other environments as well.”
Ori Goshen Sep 20, 2023 ▶ 24:51
Assertion Not checkable as stated
Goshen: 95% of enterprise AI use cases are task-specific
“95% of the use cases in the enterprise are actually pretty specific.”
Ori Goshen Sep 20, 2023 ▶ 26:43
Prediction Not checkable as stated
Goshen: Enterprise AI will not be a winner-take-all market
“I don't think it's a winner take all kind of market. And I don't think it's a single approach that will be a winner here.”
Ori Goshen Sep 20, 2023 ▶ 30:14
Opinion
Goshen: Chat interfaces are not optimal for consuming and producing information
“And it's not delivered in a chat interface. And we did it deliberately because we haven't, we don't think chat interface is definitely the most optimal way to consume and kind of produce information.”
Ori Goshen Sep 20, 2023 ▶ 34:24
Assertion Supported
Goshen: AI21 Labs' Wordtune has over 10 million users
“So we launched it three years ago and it already has more than ten million users and it's continuously growing.”
Ori Goshen Sep 20, 2023 ▶ 34:57
Prediction Not checkable as stated
Goshen: Enterprise generative AI will see production rollouts in six months
“I think in, in six months, we'll see gradual rollouts to production for many use cases.”
Ori Goshen Sep 20, 2023 ▶ 40:58
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more