Ben Rogojan: API connections are usually the hardest part of data extraction
Ben Rogojan (Seattle Data Guy) · Understanding Data Engineering in 2025 | Ben Rogojan, Seattle Data Guy · Jan 23, 2025 · at 33:38
Ben Rogojan (Seattle Data Guy), a data engineering consultant, discusses why automated data extraction and ELT workflows remain complex and expensive.
0:00 / 0:04exact quote · 4.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Cause that's usually the big thing. It's like the API connection tends to be the one that is the hardest.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Ben Rogojan (Seattle Data Guy)
Rogojan: Apache Iceberg is becoming the de facto open table format standard
“Iceberg as the de facto, like if we're going to use an open, open table format, that's going to be likely the one people will, I assume, pick since everyone's now supporting it, right?”
Rogojan: SQL will remain essential as LLM raw data-dumping approaches fail
“The next one I said SQL isn't going anywhere, and I kind of said that for more than one reason. One, I feel like someone's going to tell us it's time to revamp data lake v one again, where we'll just dump everything in there and we'll let an LLM figure it out …”
Ben Rogojan: Data teams bear primary responsibility for bridging business communication gaps
“I think in general, I found that if you're the data team, you're going to probably have to do a little more of the lift just because it's, Maybe a little easier for you to make that leap.”
Rogojan: Fully replacing data engineers with AI is still years away
“So I think we're a few years away from just like replacing data engineers, which has been the goal for what it feels like a decade or plus now.”
Rogojan: Databricks and Snowflake are fighting for workflows, not contract lock-in
“It's really been a white fight for workflows versus a fight for like contracts. Cause in the past it was like, okay, you sign a ten million dollar contract for Teradata. You're kind of stuck with it, right? Like there's no choice now for the next three to five…”