Jul 14, 2024 · 1h 19m · lennys-podcast
Building a world-class data org | Jessica Lachs (VP of Analytics and Data Science at DoorDash)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this deep-dive interview, DoorDash VP of Analytics and Data Science Jessica Lachs outlines her masterclass framework for building a centralized, high-impact data organization, establishing simple input metrics, and fostering a culture of extreme operational ownership.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Lenny holds 23.1% of the talking time here. How this is scored →
Lenny as informed peer 3.1 Guest teaching 4.1 Guest disagreement 0.3 Lenny pushing back 0.1
speaking balance: gold is Lenny, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 5:25 Firm rejection of embedded data team structure
Jessica takes a forceful contrarian stance, directly disagreeing with industry proponents of embedded analytics and asserting the superiority of a centralized model.
Hardest push from Lenny ▶ 6:36 Probing reporting lines versus goal settingLenny stops Jessica to challenge and clarify the precise structural mechanics of centralized versus embedded setups regarding operational control.
Biggest teaching moment ▶ 44:45 Debunking retention as a workable target metricJessica educates the host on why retention is fundamentally flawed as a short-term experimentation goal and outlines how to use proxy inputs instead.
Lenny holds their own ▶ 52:57 Airbnb healthy host composite failure parallelLenny demonstrates deep product expertise by detailing Airbnb's failed 6-factor host quality score to validate Jessica's critique of composite metrics.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Lenny as informed peer | Guest teaching | Guest disagreement | Lenny pushing back | Why |
|---|---|---|---|---|---|---|
| Welcoming Jessica and Framing Data Org Design | 4 | 0 | 0 | 0 | Lenny opens by framing DoorDash's marketplace complexity across 3-4 sides and introduces Jessica's contrarian views on org design. Jessica expresses enthusiasm without disagreement. | |
| The Centralized vs. Embedded Data Org Philosophy | 3 | 5 | 2 | 1 | Jessica forcefully rejects the popular embedded analytics model, explaining why a centralized center of excellence provides superior talent consistency and alignment. Lenny asks clarifying questions on reporting lines versus goaling. | |
| Earning a Seat at the Table as a Business Partner | 3 | 6 | 1 | 0 | Lenny clarifies that central teams shouldn't become isolated Jira ticket queues. Jessica strongly agrees, detailing the exact mechanisms her team uses to earn a proactive seat at the strategy table. | |
| Team Identity and the 'A-Team' Culture | 4 | 2 | 0 | 0 | Lenny connects Jessica's culture points to Riley Newman's 'A-Team' branding at Airbnb. Jessica relates and explains how they intentionally protect time for deep exploratory work via hackathons. | |
| Case Study: Cupcake Testing and Referral Fraud | 2 | 6 | 0 | 0 | Jessica recounts a concrete case study of self-directed exploratory work involving cupcake orders that exposed bimodal distributions and referral fraud. Lenny listens and validates the takeaway. | |
| Ruthless Prioritization and Communicating Trade-Offs | 4 | 4 | 0 | 1 | Lenny asks for tactical advice on pushing back against urgent inbound requests. Jessica outlines ruthless prioritization by explicitly communicating trade-offs to business partners. | |
| Hiring Top Data Talent: Evaluating Curiosity | 3 | 6 | 0 | 0 | Jessica describes how she evaluates innate curiosity and soft skills during case interviews, specifically testing how candidates respond when told their assumptions are wrong. | |
| Jessica's Non-Traditional Path to Data Leadership | 2 | 4 | 0 | 0 | Lenny highlights Jessica's unconventional background in art and finance. Jessica candidly explains how being self-taught in SQL/Python and business-focused created a strong partnership with technical specialists. | |
| First-Principles Problem Solving and Imposter Syndrome | 2 | 3 | 0 | 0 | Jessica discusses managing imposter syndrome by solving the immediate problem in front of her from first principles rather than overthinking long-term org design. | |
| Early DoorDash Grit: Boston Launch and Outage Triage | 2 | 4 | 0 | 0 | Jessica shares war stories from DoorDash's early days, including handing out promo cards in freezing Boston mornings and doing support delivery runs during major platform outages. | |
| Customer Empathy Through the WeDash Program | 3 | 3 | 0 | 0 | Lenny references DoorDash's mandatory employee dashing program. Jessica describes how 'WeDash' builds deep cross-stakeholder empathy and serves as an effective bug catching mechanism. | |
| Extreme Ownership and Qualitative Customer Research | 3 | 5 | 0 | 0 | Lenny asks how to cultivate extreme ownership across teams. Jessica explains that data scientists are expected to conduct qualitative customer phone calls when quantitative numbers fail to explain unexpected test results. | |
| Core Principles for Defining Effective Metrics | 3 | 7 | 2 | 0 | Jessica explains metric definition fundamentals, bluntly stating that retention is a terrible metric to goal on and warning against complicated composite scores with arbitrary weights. | |
| Establishing a Common Currency for Business Decisions | 5 | 5 | 0 | 0 | Lenny compares DoorDash's metric translation to Airbnb's use of 'nights booked' as an underlying currency. Jessica details how DoorDash translates pricing, delivery speed, and merchant additions into gross order value. | |
| Simplifying Metrics: The Merchant Health Example | 6 | 4 | 0 | 0 | Jessica illustrates metric simplification using a failed 0-1 composite 'merchant health score'. Lenny jumps in with his direct experience at Airbnb where their 'healthy host' composite ran into identical operational roadblocks. | |
| Measuring Fail States and Eliminating Edge Cases | 4 | 6 | 0 | 0 | Jessica emphasizes setting concrete reduction goals on rare edge cases like 'never delivered' orders and login drop-offs. Lenny helps tease out why standard averages systematically obscure high-churn fail states. | |
| Scaling Analytics Globally Across Wolt and DoorDash | 2 | 4 | 0 | 0 | Lenny asks about running analytics globally after acquiring Wolt. Jessica explains that consumer and dasher mechanics are remarkably similar globally despite regulatory and currency nuances. | |
| AI Corner: Democratizing SQL with Ask Data AI | 2 | 3 | 0 | 0 | Jessica describes 'Ask Data AI', an internal AI tool built on top of their historical analytics office hours to help non-technical team members write and modify SQL queries independently. | |
| Building Diverse, Cross-Disciplinary Data Teams | 3 | 4 | 0 | 0 | Jessica highlights the value of cross-disciplinary hiring, explaining how bringing in talent from ops, finance, economics, and consulting creates a richer, more collaborative data team. | |
| Lightning Round: Media, Mentors, and DoorDash Milestones | 2 | 1 | 0 | 0 | Lenny conducts the lightning round covering books, television, Korean sunscreen, career mentors, and key moments when Jessica realized DoorDash had become a mainstream success. |
Statements from this episode (29)
Lachs: Analytics must drive business impact, not act as a support desk
“The first is I believe that analytics should have a seat at the table, just like engineering and product and sort of the business folks, the operators. For me, analytics is a business impact driving function and not purely a service function.”
Centralized analytics teams are superior to embedded analysts, says Lachs
“I think there are people out there who think that analytics should be embedded into business units. I strongly disagree. I believe a central model, a center of excellence is superior and I'm happy to talk about why, but that's something that I feel quite stron…”
Lachs: Centralized data teams must share partner business teams' goals
“I think on the goal side, that is something where we have the same goals that our partner teams have. And I think that that's actually an important part of a successful central model.”
DoorDash structures analytics centrally while podding members directly to partner teams
“For us, we have a central analytics team, but we are, we're divided up into pods that map perfectly with how product engineering, operations, marketing are structured as well. And so our team de facto has these Folks embedded with our partner teams, even thoug…”
Lachs: Data teams earn their influence by surfacing proactive business opportunities
“We get the seat at the table and we need to earn it by bringing opportunities that we all can go and go after.”
Lachs: Centralized data orgs maintain higher, more consistent hiring bars
“So the first thing is a consistent and high talent bar. I think this is something I saw when we would have some sort of pockets of analytics folks embedded is the having a consistent bar for talent in terms of what we're looking for. What are the technical ski…”
Lachs: Centralized data orgs boost retention through internal mobility
“Number two is actually growth opportunities. So if you're siloed, you may be the most senior data person within keep picking on marketing. But you might be the most senior sort of data scientist within marketing. Where do you go from there? I think when you ha…”
Lachs: Centralized data orgs unify metrics and prevent redundant models
“The third thing is just consistency of methodologies and metrics. So you don't have sales that was as defined by one team and sales as defined by another team. You just have. Sales and everybody is using kind of the same metrics, the same methodologies, and yo…”
Lachs: Analytics teams must set explicit goals to protect exploratory research
“You have to be very intentional to carve out time for exploratory work for deep dives, because as you mentioned, there are always more questions and more work to be done than hours in the day. And so I think being intentional about it and setting goals for you…”
DoorDash analytics team ran hackathons to protect exploratory research time
“We would do hackathons for our team to carve out days to just go and look into these really interesting things and find opportunities.”
Lachs: DoorDash Referral Payback Was Bimodal Due to Online Fraud and Couponing
“And what we noticed was that referral as a channel was a bit misleading when you would look at the average in terms of payback and that it was really a bimodal distribution. And you had one group of really great consumers who were referring other really great …”
Lachs: Making trade-offs visible gets stakeholders to drop non-essential asks
“Sometimes people don't necessarily realize the trade-offs and when you make them apparent and you put them front and center, they realize that, oh, actually, you know what? That, that asset's not important. That can wait. So I think that that's definitely some…”
Lachs: Curiosity Cannot Be Taught to Team Members
“You can't teach curiosity. Or at least I haven't found a way to do it.”
Lachs: DoorDash uses real company historical problems for early interview business cases
“Our interview process has in the early stages, a coding exercise. So we do our technical screen and a shortened version of a business case. So real world problem solving. Typically it's something actually from DoorDash history, like a real problem that we had …”
Candidate reaction to being corrected is a crucial hiring signal
“Seeing how people react to being told they're wrong is, is a really important signal in my opinion. Seeing how people respond, how they're able to take new information and kind of pivot, how they're able to make a decision”
Lachs: Non-Technical Data Leaders Keep Deep Technical Teams Focused on Business Impact
“I think that that non-traditional background has been a great thing because I'm able to hire people who have the. Technical skills that I don't have the folks with PhDs and statistics and the data scientists, machine learning and otherwise, you know, I, I'm ab…”
Lachs: Solving immediate problems beats prematurely designing global orgs
“And then I think if you think about things
From first principles about what you need right now in front of you to unblock yourself or solve a problem.
And you just focus on that instead of thinking about like, you know, a global org that you're trying to build…”
DoorDash's early Palo Alto headquarters was housed in an animal hospital
“The first time I ever went to the office headquarters in Palo Alto, which at the time was in an animal hospital the first time I went there was a huge site outage, and the whole company, it's like, 20 people at the time, you know, the whole company jumped onli…”
Lachs: DoorDash employees must do deliveries or support four times yearly
“We have a program, a WeDash program, and Keith Yandel, who's our chief business officer, did your podcast last year, and he talked about this but four times a year, all the employees go out and go dashing or do customer support”
Lachs: Retention is a terrible target metric for rapid experimentation
“Retention is a terrible thing to goal on because it's like it, it's almost impossible to drive In a meaningful way in the short term, and yet you want to be able to experiment and iterate quickly.”
Lachs: Simple, intuitive metrics outperform complex composite scores
“If people understand it, if they have an intuition around it, if it's something that people can talk about across the company, it's going to be a much better metric in terms of driving real outcomes than your made up composite score that nobody understands.”
Lachs: Translating metrics into a common currency enables cross-functional trade-offs
“It's important to understand how metrics across the company equate to one another. And so we spend a lot of time quantifying things in terms of a common currency. So for example, if I were to lower price by a dollar, what would I get in terms of, we'll say vol…”
DoorDash uses Gross Order Value and volume as common decision currency
“So, I mean, we measure things in terms of GOV so gross order value and also volume.”
Rachitsky: Rotating teams across different metrics is highly inefficient
“Rotating between different metrics is so not efficient because you get good at, we're going to move this metric and your team's like, cool, we totally understand this lever, like cancellation rate. We've become really smart at cancellation rate. And then three…”
Teams must goal against edge cases and fail states, not just averages
“With metrics, we're often looking at the average, and I think we talked about this a little bit earlier, but making sure that you're looking at the edge cases and your fail states is also really important. And so we often will set goals actually and create met…”
Lachs: DoorDash has a cross-functional team tasked with eradicating 'never delivered' orders
“So we have part of our quality analytics team and we have product engineering and ops on it as well. Their goal is to eradicate never delivered.”
Lachs: Standard metrics understate churn because lost future orders are not observed
“When you have things that cause churn, you're losing all of that consumer's subsequent orders and that. Is not necessarily observed. You're just seeing one bad experience. You're not seeing all of the lost orders because they're lost. And so I think that somet…”
DoorDash builds internal AI tools to empower non-technical staff with SQL
“Working to build these tools that will help not just our team in terms of time saving, And also to be honest, folks, folks are gonna use it on our team, but really to be able to empower non-technical users to be able to do things on their own and not have to t…”
Lachs: DoorDash analytics is a net importer of internal company talent
“What I didn't mention is how many folks we've actually had join the analytics team. From partner teams. So whether that was from engineering or from our ops team or marketing or finance, we've had a lot, we've actually had a lot more import. We are a net impor…”