Lowin: Modern tools let analysts query data warehouses directly over static CSVs
Jeremiah Lowin · Fireside Chat: Jeremiah Lowin (Prefect), Tristan Handy (dbt) with Matt Turck (Partner, FirstMark) · Dec 11, 2020 · at 4:06
Jeremiah Lowin (Founder/CEO of Prefect) compares legacy business intelligence workflows to modern data stack capabilities at Data Driven NYC.
0:00 / 0:28exact quote · 28.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“If we think back to BI tools, say five, certainly 10 years ago, you would sort of beg for a CSV and God help you if the data, if the insight you weren't, the insight you were looking for was not in that CSV, you were sort of screwed. But now thanks to, you know, great tooling and insight and standardization and education and communication that person is now empowered to actually potentially go directly into the data warehouse if desirable or interface with the folks who can provide that.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Jeremiah Lowin
Lowin: Managed hosting for open-source tools is a bad business model
“I think that that is a bad business model. And I sort of, I'm on the record of that. And for a very simple reason, which is that A company that is solving a problem needs to find the correct way to express its knowledge and its solution to that problem.”
Lowin: Prefect's functionality is a superset of Apache Airflow's
“Now it happens that our functionality is a superset of Airflow's functionality.”
Jeremiah Lowin: Every data system eventually requires defining a schema
“You're going to have a schema. It's just a question of whether you define it upfront or you figure it out later.”
Lowin: Prefect acts as insurance, delivering value when data pipelines break
“That's why we frequently describe our product as an insurance product is because we deliver value mainly when things go wrong. And this will sound crazy maybe to say, but if everything goes the way one of our users expected it to go, they really don't Need our…”
Lowin: Negative engineering is defensive work ensuring code actually runs
“And so we've named this problem, the negative engineering problem, because it's not about what you're trying to achieve. It's about all the defensive work you have to do to make sure that it actually took place, in fact, took place.”