Feb 1, 2021 · 25m · mad
Fireside Chat: Wes McKinney (Founder & CEO, Ursa Computing) with Matt Turck (Partner, FirstMark)
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this Data Driven NYC fireside chat, host Matt Turck interviews Wes McKinney, Founder and CEO of Ursa Computing, about the creation and impact of Pandas, the engineering rationale behind Apache Arrow, and the business models supporting open-source software development.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 21.2% of the talking time here. How this is scored →
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Wes openly criticizes dominant big data engines like Spark, stating that they fail to scale down efficiently to single nodes and perform slower than pandas on single-node workloads.
Hardest push from Matt ▶ 11:42 Matt challenging the necessity of Arrow over ODBC/JDBCMatt directly challenges the fundamental premise of needing Apache Arrow by asking why standard ODBC and JDBC connectors cannot simply extract data from databases.
Biggest teaching moment ▶ 11:53 Wes explaining row-oriented database connector bottlenecksWes explains the deep technical limitations of ODBC/JDBC protocols for bulk data transfer, using the memorable analogy of trying to drink a thick milkshake through a small straw.
Matt holds his own ▶ 14:28 Matt's explanation of cloud data warehouse transport bottlenecksMatt steps beyond standard interviewer prompts to outline a sophisticated hypothesis regarding data movement costs between Snowflake and ML tools, earning validation from Wes.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The Vital Role of Pandas in Data Preparation and Cleaning | 4 | 5 | 1 | 1 | Matt sets up educational prompts about pandas, ML data preparation, and dataframe terminology. Wes explains the origins of dataframes from R and S+ and how pandas series and dataframes structure tabular data. | |
| Why Python Became the Dominant Data Science Language | 3 | 5 | 1 | 1 | Matt asks why Python became the dominant data science language over alternatives. Wes clarifies that Python's dominance was not predetermined and details how a perfect storm of open-source tools enabled rapid prototyping. | |
| The Genesis and Core Mission of Apache Arrow | 2 | 5 | 1 | 0 | Matt prompts Wes on the origins of Apache Arrow. Wes educates the audience on the 2015 data interoperability crisis across cloud data lakes and big data compute engines. | |
| Bridging Databases and Data Science Ecosystems | 6 | 7 | 2 | 5 | Matt pushes back by asking why existing database connectors like ODBC and JDBC are not sufficient. Wes explains how row-oriented protocols create a severe throughput bottleneck, comparing it to drinking a thick milkshake through a small straw. | |
| How Apache Arrow Eliminates Data Conversion Bottlenecks | 7 | 4 | 1 | 2 | Matt demonstrates strong domain knowledge by offering a detailed explanation of data movement bottlenecks between cloud data warehouses like Snowflake and machine learning frameworks. Wes agrees and expands on native Arrow export support across major data warehouse vendors. | |
| Commercializing Open Source: From Ursa Labs to Ursa Computing | 4 | 3 | 1 | 1 | Matt asks about the transition from Ursa Labs as a non-profit consortium to Ursa Computing as a venture-backed commercial business. Wes explains the funding dynamics and the necessity of commercial backing for open-source scale. | |
| Audience Q&A: Apache Arrow vs. the Databricks Stack | 3 | 6 | 3 | 1 | Matt relays audience Q&A regarding Databricks/Spark and the future of parallel computing. Wes offer direct criticism of Spark's architecture, noting that it fails to scale down efficiently to single-node computing compared to pandas. |