Pinterest manages open-source data tech in-house to limit downtime
Dave Burgess · Fireside Chat: Dave Burgess (Head of Data Engineering, Pinterest) w/ Matt Turck (Partner, FirstMark) · Apr 5, 2021 · at 20:29
Dave Burgess, Head of Data Engineering at Pinterest, explains Pinterest's policy of managing core open-source infrastructure directly rather than relying on vendors.
0:00 / 0:24exact quote · 24.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Yeah, we do. I mean, we're fortunate to have the number of engineers to do that, and one of the reasons why we work directly either with the open source community or the companies that are also working on that is that we want Pinterest to be up all the time, and so if there's a problem, we want our engineers to be able to go and look at the code and be able to fix it and deploy it into production. And have very, very little downtime.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Dave Burgess
Pinterest visual search beats Google and Bing in head-to-head tests
“You'll find that Pinterest is usually the one that comes out of top.”
Apache Kafka was built on prior engineering work at Yahoo
“Things like Kafka eventually came out of LinkedIn that was based on work that we had been doing at Yahoo”
Pinterest embeds machine learning engineers directly within product teams
“With machine learning is actually distributed everywhere. We have Many, many machine learning engineers and many, many use cases. And those mission machine learning engineers are embedded within each organization.”
Pinterest wants non-coding data scientists to deploy ML models to production
“And to get to a point where we can have people that are data scientists that don't even code, that can just build models and be able to deploy those to production. So that's really what we want to get to within Pinterest as well.”