Dehghani: Enterprise data lake migration should start with data consumer needs
Zhamak Dehghani · Fireside Chat: Zhamak Dehghani (Founder, Data Mesh) with Matt Turck (Partner, FirstMark) · Oct 27, 2021 · at 26:41
Zhamak Dehghani, creator of Data Mesh and Director of Emerging Technologies at ThoughtWorks, outlines the strategy for migrating off legacy centralized data lakes.
0:00 / 0:12exact quote · 12.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think you start going backward from your consumers of the data data lakes. So look at who's accessing it, why they're accessing it, what data do they need? Work backward and go back to the source.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Zhamak Dehghani
Dehghani: Centralized data architecture creates systems fragile to macro-level change
“Centralization of the data, centralization of the organization, functional division between the data and non-data, those are the things that have led to kind of a system that is fragile to scale and change at the macro level, not at the bits and bytes level, r…”
Dehghani: Current Data Mesh setups are Frankenstein creations stitching existing tech
“At this point in time, it looks like a Frankenstein creation because we have to stitch together a lot of technologies that exist and they weren't designed for this model of reconfiguring, you know, being reconfigured in this way.”
Dehghani: Data engineering and data science will become basic engineering skills
“I think one of the big changes would be, we'll move from this specialized and specialization to generalization. So some of the things that we consider specialization today, like data engineering, a large portion of what we call data science becomes basic engin…”
Dehghani: The debate between data warehouses and lakes is irrelevant
“And I think this kind of funny war between warehouse and lake and web model We should access the data that that seems to me a little bit irrelevant because both of those access models are very acceptable.”
Dehghani: Enterprise data usage shifted from operational reporting to embedding ML in applications
“We've moved away from, okay, I'm going to run a few, set up a warehouse and get a few reports and get an insight into the operation of my organizations to actually I want to run, you know, include ML, a data-driven way of solving problems into every feature of…”