Airflow
product on 3 shows · 9 statements across 7 episodes
Latent Space 
A Product Market Fit Show 
the MAD Podcast
9 statements about Airflow, every show
Parakhin: Airflow is poorly suited for rapid ML experimentation compared to Tangle
“Airflow is great, but Airflow is more about you have something and you want to repeatedly run it in production on schedule. It's less about you as a team developing things and being able to share and you grabbing the standard pipeline and saying, hey, I want t…”
Airflow shifted from analytics dashboards to mission-critical operational workloads
“In the early days, Airflow was used for very, what I call analytics workloads, as in the sense that it would move data around, and ultimately, the end result would be a number in a dashboard... What ended up happening is Airflow still does those analytics work…”
Parekh: Airflow Logged 300M+ Downloads and 80K+ Company Users Including OpenAI
“Airflow has downloaded over three hundred million times last year, and there's 80,000 plus companies using it. And that includes everyone from Siege State Startups to The biggest banks in the world to companies like OpenAI and Anthropic that are building incre…”
Riot Games deployed machine learning models using Apache Airflow
“One of the teams at Riot Games was like deploying their ML models and they were using Airflow.”
Ben Rogojan: Non-Big Tech data engineers must manage their own pipeline tooling
“At Facebook, you essentially have your internal airflow that a different team runs, and you can just push files to it and it picks up the data pipelines, you know, automatically. Whereas in most other companies, you might be the person that has to even manage,…”
Apache Airflow was created in 2014 at Airbnb
“Airflow was founded in 2014 at Airbnb, and it's an absolutely viral open source project.”
Prefect founder Jeremiah Lowin launched it after facing constraints changing Airflow
“Jeremiah was a primary contributor to Airflow, wanted to make some changes, wasn't able to do so, and went and started went and started Prefect.”
Kedro leaves pipeline scheduling and failure monitoring to Airflow and Dagster
“How, what time will this pipeline run? How will I know if it failed? We'll leave those tools to Dagster, Airflow, Prefect, Luigi, and many others to actually handle for you, because they do that really well.”
Douetteau: Airflow and JupyterLab don't go far enough for enterprise data science
“Those two projects are, like, very interesting, but to some extent don't go far enough, because what you really want, at the end of the day, is the ability to capture, within the platform, everything related to, ah, to your data science project.”