Data engineering complexity is caused by bad tooling obfuscating the process
DeVaris Brown · Fireside Chat: DeVaris Brown (Founder & CEO, Meroxa) with Matt Turck (Partner, FirstMark) · Jun 21, 2021 · at 15:51
DeVaris Brown, Founder and CEO of Meroxa, discusses the core responsibilities of data engineers during a fireside chat with FirstMark Partner Matt Turck.
0:00 / 0:22exact quote · 22.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If you really break down what a data engineer is, at the end of the day, that their job is to get data from one place to the next, right? So all you really need to know is three things. Where my data's coming from, where my data's going, and what format does it need to be when it gets there, right? Like that seems pretty simple. It's just the tooling that we have in between just obfuscates that it makes it like a really, really shitty job, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from DeVaris Brown
Open-source data tool creators fail to prioritize product experience
“The open source ecosystem exists, But those people don't necessarily think about like the product experience around how does this technology actually interact with, you know, the goals that people are trying to achieve.”
Meroxa's workforce is 60% Black/Brown, 40% women, and 90% underrepresented executives
“Right now we're, 60% black and brown. 40% women. Our executive staff is 90% underrepresented. We got one white guy out of 10 people.”
Founders who only think about team building after starting are too late
“If at the point you are only thinking about building a team when you start a company, you're already too late.”
Running the JVM at scale is a pain compared to Golang
“Running the JVM at scale is a pain in the ass, and Golang is much more research efficient, and so you can do all these cool things from a single binary.”
Brown: Data tooling market consists mostly of batch point solutions
“The tooling landscape looks like a bunch of point solutions that run in batch and not in real time.”
Meroxa bypasses data warehouses to transform data on every real-time event
“What we do this differently is that we act off of every single event. So every time there's a database transaction, somebody calls your API, they interact with your website that, that makes the API call that talks to the database. We can then transform that da…”