Disclosure certainty 4/5 debate potential 1/5

Netflix data scientists are free to choose their own tools

Savin Goyal · Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark) · Feb 17, 2021 · at 19:51

Savin Goyal, Tech Lead for Netflix's ML Infrastructure Team, explains how Netflix's corporate culture influences software adoption among internal data science teams.

0:00 / 0:12exact quote · 12.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Netflix has this really interesting corporate culture of freedom and responsibility. Which means that our data science teams, they are essentially free to use whatever tooling that works best for them.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

More from Savin Goyal

Disclosure
Netflix's Metaflow is not intended to be a workflow orchestrator
“Metaflow does not intend to be a workflow orchestrator.”
Savin Goyal Feb 17, 2021 ▶ 13:46 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Disclosure
Netflix's machine learning infrastructure team consists of six engineers
“My team has six engineers right now.”
Savin Goyal Feb 17, 2021 ▶ 2:28 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Assertion Not checkable as stated
Netflix uses S3, Spark, Presto, and Snowflake for data querying
“We use SG as so like the storage layer for our data warehouse, and we use Spark, Presto, Snowflake as our query engines.”
Savin Goyal Feb 17, 2021 ▶ 3:23 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Assertion Not checkable as stated
Netflix uses open-source Titus for container orchestration
“In terms of compute our container orchestration platform is called Titus, which is yet another open source project.”
Savin Goyal Feb 17, 2021 ▶ 3:55 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Assertion Not checkable as stated
Netflix drives ETL and ML pipelines using internal scheduler Meson
“We have a workflow scheduler called Mason. Mason, which is a program scheduler that's being used to drive all of our ETL as well as machine learning pipelines.”
Savin Goyal Feb 17, 2021 ▶ 4:29 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Assertion Not checkable as stated
Most Metaflow users adopt it to avoid building ML infra teams
“A big majority of our users are essentially companies who have made serious investments in machine learning. But for one reason or the other, they don't want to invest too much into machine learning infrastructure per se.”
Savin Goyal Feb 17, 2021 ▶ 22:37 Fireside Chat: Savin Goyal (ML Infra team (Metaflow), Netflix) with Matt Turck (Partner, FirstMark)
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.