StarZero All shows Try the podcast clipper →

Oct 27, 2021 · 32m · mad

Fireside Chat: Zhamak Dehghani (Founder, Data Mesh) with Matt Turck (Partner, FirstMark)

Zhamak Dehghani · 23m spoken Matt Turck · 6m spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

In this Data Driven NYC fireside chat, ThoughtWorks Director of Emerging Technologies Zhamak Dehghani discusses the core principles, technical implementation challenges, and future vision of Data Mesh with host Matt Turck. She articulates how decentralizing data ownership and treating data as a product solves the scaling and synchronization bottlenecks inherent in traditional centralized data lakes and warehouses.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 21.7% of the talking time here. How this is scored →

Matt as informed peer 2.6 Guest teaching 3.0 Guest disagreement 0.8 Matt pushing back 0.6
05100:0010:0020:0030:003:06–5:34 · Matt as informed peer 2/10 Failures of Centralized Data Architectures Matt sets up the conversation by asking what the industry is currently doing wrong regarding data architecture complexity. Zhamak references Matt's own published data ecosystem landscape map, agreeing that low-level scale has been solved while organizational scale remains fragile due to centralization.5:34–9:55 · Matt as informed peer 1/10 Core Principles of Data Mesh Matt asks Zhamak to define Data Mesh directly. Zhamak outlines the core sociotechnical principles of decentralization, data as a product, federated governance, and self-serve platforms, acknowledging that technical compromises in this model often draw pushback from traditionalists.9:55–21:04 · Matt as informed peer 5/10 Defining "Data as a Product" and Current Implementation Challenges Matt drives the conversation deeper into implementation mechanics, asking whether central catalogs violate decentralization and whether data products function like SLAs. Zhamak reframes data from passive storage into active 'data quanta' with native agency and admits current implementations look like a 'Frankenstein creation' stitched from legacy tools.21:04–26:07 · Matt as informed peer 3/10 Industry Gaps, Needed Standards, and the 10-Year Vision Matt asks whether Data Mesh can be achieved with existing tooling or if new industry standards are required. Zhamak explains necessary gaps around bitemporality, immutability, and access control standards, while envisioning a 10-year shift from specialized data engineers to generalist software engineers.26:07–31:53 · Matt as informed peer 2/10 Audience Q&A on Migration, Immutability, and Resources Matt moderates audience Q&A covering migration paths from data lakes, immutability joins, and upcoming educational resources. Zhamak delivers an expert explanation on using dual timestamps (event time and processing time) to enable joins on immutable data.3:06–5:34 · Guest teaching 2/10 Failures of Centralized Data Architectures Matt sets up the conversation by asking what the industry is currently doing wrong regarding data architecture complexity. Zhamak references Matt's own published data ecosystem landscape map, agreeing that low-level scale has been solved while organizational scale remains fragile due to centralization.5:34–9:55 · Guest teaching 2/10 Core Principles of Data Mesh Matt asks Zhamak to define Data Mesh directly. Zhamak outlines the core sociotechnical principles of decentralization, data as a product, federated governance, and self-serve platforms, acknowledging that technical compromises in this model often draw pushback from traditionalists.9:55–21:04 · Guest teaching 4/10 Defining "Data as a Product" and Current Implementation Challenges Matt drives the conversation deeper into implementation mechanics, asking whether central catalogs violate decentralization and whether data products function like SLAs. Zhamak reframes data from passive storage into active 'data quanta' with native agency and admits current implementations look like a 'Frankenstein creation' stitched from legacy tools.21:04–26:07 · Guest teaching 3/10 Industry Gaps, Needed Standards, and the 10-Year Vision Matt asks whether Data Mesh can be achieved with existing tooling or if new industry standards are required. Zhamak explains necessary gaps around bitemporality, immutability, and access control standards, while envisioning a 10-year shift from specialized data engineers to generalist software engineers.26:07–31:53 · Guest teaching 4/10 Audience Q&A on Migration, Immutability, and Resources Matt moderates audience Q&A covering migration paths from data lakes, immutability joins, and upcoming educational resources. Zhamak delivers an expert explanation on using dual timestamps (event time and processing time) to enable joins on immutable data.3:06–5:34 · Guest disagreement 0/10 Failures of Centralized Data Architectures Matt sets up the conversation by asking what the industry is currently doing wrong regarding data architecture complexity. Zhamak references Matt's own published data ecosystem landscape map, agreeing that low-level scale has been solved while organizational scale remains fragile due to centralization.5:34–9:55 · Guest disagreement 1/10 Core Principles of Data Mesh Matt asks Zhamak to define Data Mesh directly. Zhamak outlines the core sociotechnical principles of decentralization, data as a product, federated governance, and self-serve platforms, acknowledging that technical compromises in this model often draw pushback from traditionalists.9:55–21:04 · Guest disagreement 1/10 Defining "Data as a Product" and Current Implementation Challenges Matt drives the conversation deeper into implementation mechanics, asking whether central catalogs violate decentralization and whether data products function like SLAs. Zhamak reframes data from passive storage into active 'data quanta' with native agency and admits current implementations look like a 'Frankenstein creation' stitched from legacy tools.21:04–26:07 · Guest disagreement 1/10 Industry Gaps, Needed Standards, and the 10-Year Vision Matt asks whether Data Mesh can be achieved with existing tooling or if new industry standards are required. Zhamak explains necessary gaps around bitemporality, immutability, and access control standards, while envisioning a 10-year shift from specialized data engineers to generalist software engineers.26:07–31:53 · Guest disagreement 1/10 Audience Q&A on Migration, Immutability, and Resources Matt moderates audience Q&A covering migration paths from data lakes, immutability joins, and upcoming educational resources. Zhamak delivers an expert explanation on using dual timestamps (event time and processing time) to enable joins on immutable data.3:06–5:34 · Matt pushing back 0/10 Failures of Centralized Data Architectures Matt sets up the conversation by asking what the industry is currently doing wrong regarding data architecture complexity. Zhamak references Matt's own published data ecosystem landscape map, agreeing that low-level scale has been solved while organizational scale remains fragile due to centralization.5:34–9:55 · Matt pushing back 0/10 Core Principles of Data Mesh Matt asks Zhamak to define Data Mesh directly. Zhamak outlines the core sociotechnical principles of decentralization, data as a product, federated governance, and self-serve platforms, acknowledging that technical compromises in this model often draw pushback from traditionalists.9:55–21:04 · Matt pushing back 2/10 Defining "Data as a Product" and Current Implementation Challenges Matt drives the conversation deeper into implementation mechanics, asking whether central catalogs violate decentralization and whether data products function like SLAs. Zhamak reframes data from passive storage into active 'data quanta' with native agency and admits current implementations look like a 'Frankenstein creation' stitched from legacy tools.21:04–26:07 · Matt pushing back 1/10 Industry Gaps, Needed Standards, and the 10-Year Vision Matt asks whether Data Mesh can be achieved with existing tooling or if new industry standards are required. Zhamak explains necessary gaps around bitemporality, immutability, and access control standards, while envisioning a 10-year shift from specialized data engineers to generalist software engineers.26:07–31:53 · Matt pushing back 0/10 Audience Q&A on Migration, Immutability, and Resources Matt moderates audience Q&A covering migration paths from data lakes, immutability joins, and upcoming educational resources. Zhamak delivers an expert explanation on using dual timestamps (event time and processing time) to enable joins on immutable data.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 50.7% · guest 49.3%0:00 · Matt 50.7% · guest 49.3%3:00 · Matt 19.2% · guest 80.8%3:00 · Matt 19.2% · guest 80.8%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 36.2% · guest 63.8%9:00 · Matt 36.2% · guest 63.8%12:00 · Matt 8.8% · guest 91.2%12:00 · Matt 8.8% · guest 91.2%15:00 · Matt 18.5% · guest 81.5%15:00 · Matt 18.5% · guest 81.5%18:00 · Matt 0% · guest 100%18:00 · Matt 0% · guest 100%21:00 · Matt 23.4% · guest 76.6%21:00 · Matt 23.4% · guest 76.6%24:00 · Matt 19.3% · guest 80.7%24:00 · Matt 19.3% · guest 80.7%27:00 · Matt 11.7% · guest 88.3%27:00 · Matt 11.7% · guest 88.3%30:00 · Matt 54.8% · guest 45.2%30:00 · Matt 54.8% · guest 45.2%
Sharpest disagreement ▶ 8:35 Making enemies over technical compromises

Zhamak explicitly highlights where her paradigm clashes with traditional data engineering views, noting that proposing true decentralization loses friends and makes enemies among architectural purists.

 Hardest push from Matt ▶ 13:08 Challenging catalog centralization

Matt presses Zhamak on whether relying on a data catalog reintroduces the exact central bottleneck that Data Mesh seeks to eliminate.

 Biggest teaching moment ▶ 13:24 Reframing data as active computational quanta

Zhamak corrects the conventional mental model of data as passive bits on disk requiring central indexing, reframing it as an active 'data quantum' with embedded discoverability APIs and code.

 Matt holds his own ▶ 15:38 Translating abstractions into software contracts and SLAs

Matt demonstrates sharp domain knowledge by synthesizing Zhamak's abstract concepts into concrete engineering terms like data product contracts, guarantees, and operational SLAs.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Failures of Centralized Data Architectures 2200 Matt sets up the conversation by asking what the industry is currently doing wrong regarding data architecture complexity. Zhamak references Matt's own published data ecosystem landscape map, agreeing that low-level scale has been solved while organizational scale remains fragile due to centralization.
Core Principles of Data Mesh 1210 Matt asks Zhamak to define Data Mesh directly. Zhamak outlines the core sociotechnical principles of decentralization, data as a product, federated governance, and self-serve platforms, acknowledging that technical compromises in this model often draw pushback from traditionalists.
Defining "Data as a Product" and Current Implementation Challenges 5412 Matt drives the conversation deeper into implementation mechanics, asking whether central catalogs violate decentralization and whether data products function like SLAs. Zhamak reframes data from passive storage into active 'data quanta' with native agency and admits current implementations look like a 'Frankenstein creation' stitched from legacy tools.
Industry Gaps, Needed Standards, and the 10-Year Vision 3311 Matt asks whether Data Mesh can be achieved with existing tooling or if new industry standards are required. Zhamak explains necessary gaps around bitemporality, immutability, and access control standards, while envisioning a 10-year shift from specialized data engineers to generalist software engineers.
Audience Q&A on Migration, Immutability, and Resources 2410 Matt moderates audience Q&A covering migration paths from data lakes, immutability joins, and upcoming educational resources. Zhamak delivers an expert explanation on using dual timestamps (event time and processing time) to enable joins on immutable data.

Statements from this episode (9)

Assertion Not checkable as stated
Dehghani: Enterprise data usage shifted from operational reporting to embedding ML in applications
“We've moved away from, okay, I'm going to run a few, set up a warehouse and get a few reports and get an insight into the operation of my organizations to actually I want to run, you know, include ML, a data-driven way of solving problems into every feature of…”
Zhamak Dehghani Oct 27, 2021 ▶ 2:27
Insight
Dehghani: Centralized data architecture creates systems fragile to macro-level change
“Centralization of the data, centralization of the organization, functional division between the data and non-data, those are the things that have led to kind of a system that is fragile to scale and change at the macro level, not at the bits and bytes level, r…”
Zhamak Dehghani Oct 27, 2021 ▶ 5:10
Opinion
Dehghani: The debate between data warehouses and lakes is irrelevant
“And I think this kind of funny war between warehouse and lake and web model We should access the data that that seems to me a little bit irrelevant because both of those access models are very acceptable.”
Zhamak Dehghani Oct 27, 2021 ▶ 12:33
Assertion Not checkable as stated
Dehghani: Current Data Mesh setups are Frankenstein creations stitching existing tech
“At this point in time, it looks like a Frankenstein creation because we have to stitch together a lot of technologies that exist and they weren't designed for this model of reconfiguring, you know, being reconfigured in this way.”
Zhamak Dehghani Oct 27, 2021 ▶ 18:04
Assertion Not checkable as stated
Dehghani: Data access control is proprietary compared to standardized API access
“Access control right now is very proprietary in data world to the platform you're stuck in compared to the API world where you have some sort of a, you know, standard.”
Zhamak Dehghani Oct 27, 2021 ▶ 23:26
Prediction Not checkable as stated
Dehghani: Data engineering and data science will become basic engineering skills
“I think one of the big changes would be, we'll move from this specialized and specialization to generalization. So some of the things that we consider specialization today, like data engineering, a large portion of what we call data science becomes basic engin…”
Zhamak Dehghani Oct 27, 2021 ▶ 24:15
Insight
Dehghani: Enterprise data lake migration should start with data consumer needs
“I think you start going backward from your consumers of the data data lakes. So look at who's accessing it, why they're accessing it, what data do they need? Work backward and go back to the source.”
Zhamak Dehghani Oct 27, 2021 ▶ 26:41
Insight
Dehghani: True data immutability requires both event and processing timestamps
“The data can be only immutable if we build two time timestamps into every single kind of representation of the data, and those two timestamps are when something actually happened, and when we process that information, our understanding of that data.”
Zhamak Dehghani Oct 27, 2021 ▶ 28:13
Disclosure
ThoughtWorks plans to publish an open-source Data Mesh reference implementation
“So we will publish that open source. We'll have an open source reference implementation. So we're working on it internally.”
Zhamak Dehghani Oct 27, 2021 ▶ 31:46
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more