The Exchanges

Q talk about Metaflow, uh, specifically, but, um, maybe before that, uh, could you, to the extent you can talk about it, could you give us a sort of a broad, uh, overview of the data stack, uh, including the machine learning Part, but not, not just that, like including analytics, what are some of the tools that I use, some of the open source project that I used across Netflix?

A Yeah, sure. Um, so when it comes to say the stack that's available to our data centers, you know, like, uh, first and foremost, it starts off with our data platform and Netflix has built a very comprehensive data stack. Uh, we are an AWS shop. Uh, so we use SG as Uh, so like the storage layer for our data warehouse, and we use Spark, Presto, Snowflake, uh, as our query engines. And, uh, we have invested pretty heavily in data discovery and data cataloging. Metacat is a Netflix project, uh, that's involved in, uh, data cataloging. So all of the data stored, um, so it gets illuminated over there. Then when it comes to, uh, query engines, as I mentioned, uh, Spark, Presto, Snowflake, Uh, those are sort of like, uh, some of the gradients that people, uh, prefer using, uh, in terms of compute, uh, our container orchestration platform is called Titus, which is, uh, yet another open source project. So, uh, from stock of containers, our, uh, internal users, uh, they'll essentially launch either their batch workloads or their services, uh, on top of Titus. And, uh, then when it comes to say workflow orchestration, uh, as with any big company, we have a bunch of different workflow schedulers. Uh, they're used internally for most of our ETL workloads. Uh, we have a workflow scheduler called Mason.

AI assessment note: “first and foremost, it starts off with our data platform and Netflix has built”

Q Great. Great. And, um, so, so Netflix decided to open source, uh, Metaflow in, in what, 2019, um, I'm always curious, like, why does a company, um, like Netflix decide to do that? And, um, what are some of the actual benefits that you have, uh, you know, derived from that decision?

A Sure. Yeah. Uh, so, so we open-sourced Metaflow in December of, and we have been working on Metaflow for the last three and a half years. So, uh, Once we sort of like, uh, had near universal adoption, uh, internally, uh, at Netflix, at that point in time, uh, we sort of like started talking about Metaflow publicly. Uh, so that would have been somewhere around like early, and that's when we, uh, actually saw that there was this gap, uh, in the kind of offerings that were available in the market. And, uh, there were a lot of folks who reached out, uh, who were curious, uh, to use Metaflow and, Netflix has invested in, um, open source projects for a really long period of time. You know, if you look at projects like, uh, chaos engineering, uh, Spinnaker, uh, so our hope was that, uh, with Metaflow, we can sort of like, again, you know, like establish the Netflix brand specifically in the machine learning infrastructure domain, um, as well as we had, uh, some amount of confidence that we had something new to offer. And not just like for us, open source wasn't only about that, you know, like here's some piece of code that we have now made public on GitHub and now you're like free to use it, uh, in whatever way, uh, you want it to. Uh, but also it was sort of like a good conduit for us to engage with the community to understand Uh, what are the use cases and pain points so that we cou…

AI assessment note: “establish the Netflix brand specifically in the machine learning infrastructure domain”

Q And what, what kind of company do you need to be? What kind of, um, requirements do you need to have to be a good fit for Metaflow or for Metaflow to be a good fit for you?

A So, uh, so Netflix is an AWS shop. Uh, so the current, uh, first class integrations that we have with Metaflow, uh, are for AWS. Uh, we are, uh, working on some projects to essentially change that situation in the near future. Uh, but by and large, as we stand today, if you want to sort of like, uh, run your machine learning workloads, uh, natively on a cloud, then, uh, we provide integrations with AWS. Uh, so that's one, um, sort of like, uh, requirement of sorts, but then you can also run Metaflow locally on your laptop, and you can still sort of like benefit from data version and can produce a little collaboration, uh, and a lot of organizations to use that for that. Um, so our focus With Metaflow's release has been to be, um, sort of like to cater to data scientists rather than machine learning infrastructure teams. So a big majority of our users are essentially companies who have made serious investments in machine learning. Uh, but, uh, for one reason or the other, they don't want to invest, uh, too much into machine learning infrastructure per se. Uh, so Metaflow sort of like provides, uh, them, uh, an easy way of adopting something that has worked at scale, uh, at Netflix. Uh, so if let's say there's an organization which wants to use Metaflow, all they need to do is just like pip install our package and point it to, uh, their AWS resources, uh, the deployment footprint…

AI assessment note: “companies who have made serious investments in machine learning”

Q And, um, so how does, how does it work? What does it do? Uh, what's the scope of the product?

A Sure. Yeah. So, you know, Metaflow ultimately at the end of the day, it's a Python package, uh, as well as an R package, because, uh, the users that we have internally, uh, they use either Python or R to get their work done. So we have to take care of both of those. And within Metaflow, we have tried to make sure that the number of concepts that they have to learn are just sort of kept to a minimum. Uh, so they can write their, uh, idiomatic Python and R code and, uh, Metaflow will then allow them to sort of like declare their compute in a dedicated cyclic graph. Uh, so very simply, you know, like say, If you have a workflow where you are, say, um, getting access to some data from a data warehouse, then you are, say, uh, transforming that data, maybe generating some features, then you are training a bunch of models. Uh, maybe then you are trying to figure out which model sort of, like, uh, performs the best amongst the group of models that you have trained, and then you're doing something with that model, maybe storing it, maybe pushing it, uh, elsewhere. Uh, all of this, this paradigm essentially can very easily fit into, uh, Workflow orchestrators work, uh, at the end of the day. So, so we allow users to really simply declare this graph and then, uh, Metaflow will take care of the execution of that graph, uh, on the laptop. Now, what usually ends up happening is.

AI assessment note: “allow users to really simply declare this graph and then Metaflow will take care of the execution”

Q And, uh, how does it do that? Like, how does it, uh, know that you, the, that you need a GPU or, um, that you should be connecting to something else? Like how does.

A So, so right now we require our users to be explicit about it. So, so now let's say, you know, you have Uh, define this tag, this craft. So let's say the first step of the craft is that you are playing around with some data, right? And, uh, maybe you need a memory intensive, uh, instance for that. So you can very simply just like put that step through in a decorator that says that, Hey, this instance, or like this step needs to run on an instance that has this much memory. And then now for the next, uh, step of your workflow, you can specify a different instance or a different resource requirements. And then Metaflow will take care of orchestrating that, um, on your compute cluster. And so, so this is sort of like more so focused on the development, uh, environment side of things. And what Metaflow does behind the scenes is that it will also sort of like cache Uh, and store, uh, all of the intermediate state so that you can go back in time and you can look at like how a certain workflow executed through any of your systems. And, uh, this also allows people to scale out, uh, their compute really, really well. Uh, an example here could be, you know, like we have users who, um, Want to say train models for every single country that Netflix operates in as well as every single language that Netflix operates in. So all of a sudden the cardinality of the number of models that you want…

AI assessment note: “So right now we require our users to be explicit about it.”

Q Very, very good. Sorry, I interrupted you. What did you, did you?

A No, no. So I was, like, trying to give, uh, a very general overview of, like, some of the capabilities that Metaflil has, but, uh, Like by and large, if you look at say the current MLOps ecosystem, uh, you know, it's, it's as big as the software installing ecosystem, uh, by itself where people have to worry about different layers of the stack. And there are a lot of great open source projects, as well as commercial vendors, uh, which are making good problems, you know, like things like, uh, how do you sort of like have efficient data access? Uh, how do you manage your compute instances when it comes to workflow orchestrator? What is the story around that when it comes to monitoring? Uh, your ML workflows, as well as your, uh, ML inferencing endpoints. Uh, there are a variety of different solutions as well. So, but then the gap is that as an end user, as a data scientist, how do you sort of like move between, uh, any of these components? Uh, you know, let's say if you're using S three as your data store, but now if you're using Kubernetes, uh, as your compute cluster and say, if you want to use step functions or Argo, As your, uh, orchestrate, uh, workflow orchestration layer. And then if you throw in some sort of like monitoring tooling, um, as, as a data scientist, then I have to sort of like, think about like all the best practices around like how do I go about writing my cod…

AI assessment note: “So I was, like, trying to give, uh, a very general overview”