May 24, 2021 · 46m · mad
Fireside Chat: Ali Ghodsi (Founder & CEO, Databricks) with Matt Turck (Partner, FirstMark)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this fireside chat hosted by Matt Turck, Databricks Founder and CEO Ali Ghodsi discusses the journey of founding Databricks from UC Berkeley's AMP Lab to scaling it into a global enterprise data and AI leader. He shares insights into the technical evolution of the Lakehouse architecture, open-source commercialization strategies, organizational scaling, and the future role of AI in enterprise software.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 19% of the talking time here. How this is scored →
Matt as informed peer 4.1 Guest teaching 5.0 Guest disagreement 1.1 Matt pushing back 1.3
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 21:05 Rejection of zero-sum Snowflake rivalry
Ali directly reframes the host's framing of a winner-take-all clash with Snowflake, comparing the dynamic to coexisting cloud providers like AWS, Azure, and Google Cloud.
Hardest push from Matt ▶ 19:18 Host probing on Lakehouse trade-offsMatt refuses to accept the Lakehouse model as purely superior without scrutiny, directly asking whether there are technical trade-offs to combining data lakes and warehouses.
Biggest teaching moment ▶ 8:50 Reframing PLG vs Enterprise Go-To-MarketAli educates the host on product-market-channel fit, explaining why relying purely on product-led credit-card growth was a strategic mistake for strategic enterprise software.
Matt holds his own ▶ 11:16 Recalling 2015 interview to trace product expansionMatt demonstrates deep background knowledge by recalling his 2015 interview with co-founder Ion Stoica and accurately mapping Databricks' transition from MapReduce replacement to full ML/Lakehouse platform.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| The Origin of Databricks and UC Berkeley's AMP Lab | 4 | 5 | 1 | 1 | Matt opens with well-informed context about UC Berkeley's AMP Lab, Spark, and academic culture. Ali educates the host on how 1970s ML algorithms achieved modern breakthroughs simply by scaling data volumes on distributed systems. | |
| Managing a Large Team of Seven Co-Founders | 3 | 5 | 1 | 1 | Matt probes into the rare dynamic of having seven co-founders and asks about scaling from 0 to 10M ARR. Ali candidly explains their early GTM misstep of relying on product-led credit-card growth rather than executive enterprise sales. | |
| Evolution of Databricks into a Multi-Product Platform | 6 | 5 | 1 | 1 | Matt demonstrates high expertise by referencing his 2015 interview with Databricks co-founder Ion Stoica and detailing the product evolution from Spark to MLflow and Lakehouse. Ali explains their AWS-inspired open-source monetization playbook. | |
| The Lakehouse Paradigm vs. Traditional Warehouses | 5 | 5 | 1 | 2 | Matt asks technical questions contrasting data lakes with data warehouses and specifically inquires about Apache Iceberg and trade-offs. Ali details how Delta Lake provided transactional structure over data lakes. | |
| Lakehouse Dominance vs. Coexistence with Snowflake | 5 | 5 | 2 | 3 | Matt raises industry chatter regarding the competition between Snowflake and Databricks. Ali reframes the zero-sum narrative by drawing parallels to major cloud provider coexistence while holding that Lakehouse architecture will win long-term. | |
| Organizational Structure for Multi-Product R&D | 4 | 5 | 1 | 1 | Matt questions how Databricks organizes R&D to avoid being a one-product company. Ali outlines their organizational separation of disruptive zero-to-one teams from enterprise maintenance teams based on 'Zone to Win'. | |
| Open Source Business Model and the Future of AI | 4 | 5 | 1 | 1 | Matt asks if Ali would start another company as open-source first. Ali argues open source hosted in the cloud is evolutionarily superior to proprietary software models and predicts AI will consume traditional software. | |
| Hybrid Go-To-Market Strategy and Developer Advocacy | 4 | 5 | 1 | 1 | Matt asks how open source fits into sales motions without internal channel conflict. Ali explains how community edition usage generates half their enterprise sales pipeline and bypasses long proof-of-concept cycles. | |
| Scaling as CEO and Executive Leadership Recruitment | 4 | 5 | 1 | 1 | Matt asks how Ali scaled personally as CEO and handles internal promotion versus external executive recruiting. Ali describes evaluating leaders on whether they can build a car from scratch rather than just drive it. | |
| Q&A: Data Mesh Architecture | 3 | 5 | 1 | 1 | Matt asks an audience question about Data Mesh architecture. Ali explains Data Mesh as an organizational decentralization response to centralized bottlenecked data teams, supported by Lakehouse governance. | |
| Q&A: Managed Cloud Containers and Kubernetes | 3 | 5 | 1 | 1 | Matt selects a technical question on cloud containers and Kubernetes. Ali compares Kubernetes to a universal hardware standard like USB, while stressing that data platforms require higher-level governance layers. |
Statements from this episode (18)
Ghodsi: Early Big Tech achieved AI breakthroughs using 1970s algorithms with massive data
“What they were doing is they were taking those algorithms from the seventies that do not work, but they were applying orders of magnitude, more data to it. So a lot of data on modern hardware, and they were getting superhuman results.”
Ghodsi: Hadoop was terrible for machine learning tasks
“The people in Amplab that were doing machine learning, the math folks, they had to use this thing called Hadoop, which was just terrible.”
Ghodsi: Databricks' early reliance on product-led growth was a mistake
“On the channel side, the mistake with it is we really early on, we're really big believers in this product led growth. We said, you know, we're going to build this beautiful simplified product that we now have. We put it online and it's going to be cloud based…”
Ghodsi: Founders cannot pick their sales channel; product-market fit dictates it
“You don't get to pick your channel. You can, you don't get to say, oh, I want my ASP to be 50 K or 60. That's not your choice. You have a product. You have a market. If it has fit, you have to find the right channel to connect those two.”
Ghodsi: Databricks open-sources products only after achieving product-market fit
“So our secret sauce is look at an enterprise problem, figure out what that is, understand it deeply by being really customer obsessed, bring the problem back, have the innovators, the PhDs that know how to solve these problems. Solve the problem. Iterate quick…”
Ghodsi: Creating open-source software gives cloud hosts a competitive advantage
“We're going to host open source software in the cloud, but the difference is we'll create open source software. That way we get competitive advantage with respect to anyone else who would want to do the same thing, right? Otherwise anyone can pick up any open …”
Ghodsi: Four major technical breakthroughs enabled the Lakehouse around 2016-2017
“Yeah, actually, the four technological breakthroughs that kind of happened at the same time, 2016, 17, at the same time, the one we contributed was Delta Lake, there was Hootie, there was Hive acid, and there was icebergs.”
Ghodsi: Open-source Lakehouses will eventually render proprietary data warehouses obsolete
“So slowly what's happening is that the open source realm An ecosystem is emerging where you can do all of your analytics in this lake house paradigm, and you don't, eventually it will be the case that you will not need all these other, you know, proprietary ol…”
Ghodsi: Snowflake coexists with Databricks in about 70% of accounts
“It's certainly going to coexist, and it already coexists with Databricks in probably 70% of the accounts we're in.”
Ghodsi: Databricks re-wrote Apache Spark's execution engine in C++
“Two or three years ago, we set out to re-implement all Spark in C++ in what we call the really, really fast, what's called MPP engine, Massive Apparel Processing Engine.”
Ghodsi: Databricks will open-source lower-stack layers and build proprietary software above
“We're definitely going to continue to move up the stack and then commoditize the stuff that's below by open sourcing it and just releasing it to the market and making it the standard and then moving up the stack with innovations.”
Databricks splits R&D into enterprise stability and new innovation teams
“All of engineering and product is separated into two different pieces. One that focuses on the things that enterprises need, large enterprises, encryption, security, authentication, stability, and so on. And another piece that focuses on these innovations.”
Ghodsi: Enterprise demand starves new innovation unless R&D teams are separated
“So you actually, or chart wise should separate those out because otherwise what happens when you're successful is that the former gets all of the resources because the big enterprises have infinite demand for your for the things that you're doing.”
Ghodsi: Any proprietary software company is ripe for open-source disruption
“Any proprietary software company out there is ripe for disruption by an open source competitor.”
Ghodsi: Half of Databricks' leads come from free Community Edition
“Half of our leads comes from that.”
Ghodsi: Databricks hires executive leaders who build, not just maintain
“What are the things we look for? We look for people who have seen build. So the joke I say is, do you have a driver's license? And people will say, I have it. Are you good at driving your car? Yes, I'm very good at it. Why are you asking? Can you build a car w…”
Ghodsi: Centralized data teams inevitably become operational bottlenecks
“As you can imagine, it would never scale in a large organization. That team would become bottlenecked. They would not understand how to prioritize the different projects because they don't understand the asks of marketing, sales, customer success and everythin…”