Dada argues AWS SageMaker is a deployment target, not a code-structuring tool
Kedro Product Manager · Introducing Kedro · Feb 17, 2021 · at 16:27
Yetunde Dada, Principal Product Manager at QuantumBlack, explains how open-source framework Kedro complements cloud platform AWS SageMaker.
0:00 / 0:19exact quote · 19.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“We use, we think of Kedra or SageMaker as a deployment target for Kedra. So still that whole process of like creating like, well document, well structured, Modular data science code, but still having the freedom to deploy it on SageMaker is how we see those roles interacting. SageMaker doesn't quite help you with those two aspects.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Kedro Product Manager
Dada argues building data pipelines is harder than machine learning itself
“And this is where we believe that machine learning is not really the hard part, but building and maintaining the data pipeline is”
Kedro leaves pipeline scheduling and failure monitoring to Airflow and Dagster
“How, what time will this pipeline run? How will I know if it failed? We'll leave those tools to Dagster, Airflow, Prefect, Luigi, and many others to actually handle for you, because they do that really well.”
Dada asserts notebooks are challenging for reproducible, production-ready data pipelines
“In terms of actually building your full pipeline and notebook that's where things get a little bit complicated because notebooks are a little bit challenging for reproducible and what we call production-ready code.”
Dada says data science experiment parameters must be separated from codebases
“So your experiment parameters for data science workflow should also be outside of your code base because we really believe it helps you make code that is generalizable and reusable by removing things that loading and load the code for loading and saving your d…”