Mar 19, 2019 · 20m · mad
Optionality in Data Architecture // Justin Borgman, Starburst Data (FirstMark's Data Driven NYC)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this presentation at FirstMark's Data Driven NYC, Starburst Data CEO Justin Borgman advocates for architectural optionality in enterprise data systems, explaining how separating storage from compute, utilizing open data formats, and establishing SQL abstraction layers prevent vendor lock-in and future-proof data infrastructure.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 1.3% of the talking time here. How this is scored →
Matt as informed peer 0.2 Guest teaching 1.5 Guest disagreement 0.5 Matt pushing back 0.0
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 9:35 Critique of unnamed proprietary database lock-in
Justin explicitly critiques a highly-funded database competitor for repeating vendor lock-in mistakes via proprietary storage.
Hardest push from Matt ▶ 12:23 Host redirects focus to company statusHost Matt Turck interrupts the transition to steer Justin toward giving concrete business metrics on Starburst's headcount and progress.
Biggest teaching moment ▶ 18:02 Explaining Spark vs Presto concurrency limitationsJustin clarifies that Spark is built for machine learning/ETL rather than high concurrency, educating the room on where Presto outperforms Spark SQL.
Matt holds his own ▶ 3:20 Acknowledgement of host's industry taxonomyJustin references Matt Turck's seminal Big Data Landscape chart, highlighting the host's recognized expertise in tracking the data ecosystem.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Speaker Background & Career History | 0 | 1 | 0 | 0 | Justin presents his background founding Hadapt and selling to Teradata. As a presentation monologue, the host is inactive, resulting in zero host expertise and zero pushback. | |
| Presto Architecture & Connectivity | 0 | 1 | 0 | 0 | Justin discusses Presto connectivity and cites Matt Turck's famous Big Data Landscape chart. The monologue nature keeps host interaction to zero while guest offers technical context. | |
| A Real-World Lesson in Lock-In | 0 | 1 | 0 | 0 | Justin illustrates vendor lock-in through a personal narrative about his Boston gas utility service being cut off. Host participation is non-existent during this slide presentation. | |
| Three Keys to Architectural Longevity | 0 | 2 | 2 | 0 | Justin details architectural principles including open file formats like ORC and Parquet, while briefly calling out proprietary database lock-in. Host scores remain zero for this presentation block. | |
| Future-Proofing with SQL Abstraction | 0 | 1 | 0 | 0 | Justin concludes his talk by emphasizing SQL abstraction layers for enterprise freedom before opening for questions. Host scores remain zero in this final monologue section. | |
| Q&A and Fireside Chat | 1 | 3 | 1 | 0 | Matt prompts Justin on company growth before opening to audience Q&A covering HIPAA, Facebook relations, and Spark concurrency. The dynamic is cooperative and informative without conflict or host pushback. |
Statements from this episode (13)
Justin Borgman: Most large enterprise banks, retailers, and media firms use Teradata
“If you work for a large bank or a large retailer or a large media company, you probably have Teradata.”
Borgman: Facebook Runs the Largest MySQL Deployment
“Facebook happens to run the largest MySQL deployment as well.”
Borgman: AWS Athena runs on Presto under the covers
“Athena actually runs on Presto under the covers, so that is the technology behind Athena.”
Justin Borgman: Specialized data tools lead to silos and vendor lock-in
“Over time, that creates a lot of data silos, and in particular, as you create these data silos, you're confronted with vendor lock-in, particularly if you're using proprietary databases or storage as your underlying infrastructure for these different use cases…”
Borgman: Modern architecture requires separated storage, open formats, and abstraction
“The three keys in my view to a data architecture that can stand the test of time are first and foremost, embrace this new, ah, frontier of storage compute separation. I'll talk about that in a moment. Use open data formats wherever possible. That is really, re…”
Borgman: Storage and compute separation provides cost flexibility through on-demand scaling
“You can leave all of your data in S three, which is essentially your lowest cost option, right? In the cloud. And you can spin up these EC two nodes as needed as demand increases for whatever workload you're doing. And as a result, you have all this cost flexi…”
Borgman: Facebook runs essentially all of its infrastructure using ORC
“Essentially Facebook runs all of their infrastructure using ORC file.”
Borgman: Netflix uses Parquet for essentially all of its infrastructure
“Ah, Parquet, Netflix uses that for all of their infrastructure essentially.”
Borgman: Adopting popular database vendors with proprietary storage repeats vendor lock-in
“And I think that's a mistake that a lot of people are making today, is there's a very popular database company right now that I won't mention, that is very, very popular, has raised a lot of money, and once again, it is that that story of vendor lock-in all ov…”
Borgman: SQL abstraction layers insulate users from underlying data changes
“My view is creating a sequel abstraction layer at least in the context of sequel analytics is a big part of the answer and by doing that if you if you use something that allows you to separate storage from compute and allows you to query any data source you ca…”
Borgman: Starburst Data is entirely bootstrapped and profitable from day one
“We haven't raised any venture capital thus far. And in fact, we're actually profitable, and we've been running profitably from day one,”
Borgman: Major global ride-sharing platforms run Presto for data analytics
“Most of the world's ride-sharing companies around the world actually run Presto for their analytics because of that, so You know, that's Lyft here in the US that's GrabTaxi in Singapore, that's OlaCabs in India, that's Kareem in the Middle East.”