Liu: Prefect and Zapier are well-positioned to build AI workflow UIs
“I think, you know, people like Prefect and Zapier have a pretty good shot at doing a good job.”
Prefect founder Jeremiah Lowin launched it after facing constraints changing Airflow
“Jeremiah was a primary contributor to Airflow, wanted to make some changes, wasn't able to do so, and went and started went and started Prefect.”
Kedro leaves pipeline scheduling and failure monitoring to Airflow and Dagster
“How, what time will this pipeline run? How will I know if it failed? We'll leave those tools to Dagster, Airflow, Prefect, Luigi, and many others to actually handle for you, because they do that really well.”
Lowin: Prefect acts as insurance, delivering value when data pipelines break
“That's why we frequently describe our product as an insurance product is because we deliver value mainly when things go wrong. And this will sound crazy maybe to say, but if everything goes the way one of our users expected it to go, they really don't Need our…”
Lowin: Negative engineering is defensive work ensuring code actually runs
“And so we've named this problem, the negative engineering problem, because it's not about what you're trying to achieve. It's about all the defensive work you have to do to make sure that it actually took place, in fact, took place.”
Lowin would have built Prefect within Apache Airflow if possible
“And if I were able to do the things that Prefect does in airflow, I would have.”
Lowin: Prefect's functionality is a superset of Apache Airflow's
“Now it happens that our functionality is a superset of Airflow's functionality.”
Lowin: Saving time during outages offers higher leverage than routine orchestration
“If you imagine saving someone one hour orchestrating something, just getting that code to run versus saving a team one hour in a production crisis incident, the leverage of that hour, it's the same hour, but the leverage, the impact, the emotional burden, the …”
Lowin: Mapping is Prefect's most popular feature
“It's by far the most popular feature of Prefect. And we can see that in our data. And that's because it solves a real problem that I experienced with data science, which I have a bunch of things I need to work with, but I don't know about it until Until runtim…”