Feb 25, 2019 · 20m · mad
CI/CD for Machine Learning & AI // Dillon Erb, Paperspace (FirstMark's Data Driven NYC)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
At FirstMark's Data Driven NYC event, Paperspace CEO Dillon Erb presents a comprehensive deep dive into CI/CD for Machine Learning and AI, contrasting traditional software delivery with the unique infrastructure and pipeline requirements of modern AI models.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 3.4% of the talking time here. How this is scored →
Matt as informed peer 0.7 Guest teaching 1.0 Guest disagreement 0.2 Matt pushing back 0.5
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 13:10 Dillon addresses competitive threat from cloud giants
Dillon acknowledges the difficulty of competing with tech giants before arguing that cloud providers view the issue as lock-in infrastructure rather than a multi-cloud software product.
Hardest push from Matt ▶ 13:04 Matt asks why a standalone company can survive against cloud providersMatt immediately confronts Dillon with the hardest venture question: why build a dedicated company when Amazon and Google continuously expand into core infrastructure.
Biggest teaching moment ▶ 17:25 Dillon details why traditional version control fails for ML hyperparameter searchesDillon educates an audience member on the breakdown of standard web tools like Git, pointing out that treating every hyperparameter search run as a Git commit shows how traditional paradigms fail ML.
Matt holds his own ▶ 13:04 Matt zeroes in on Paperspace's core strategic threatMatt demonstrates sharp venture capital expertise by directly asking how Paperspace avoids getting commoditized by cloud incumbents.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Data Driven NYC Event Title and Speaker Introductions | 0 | 0 | 0 | 0 | Dillon Erb delivers an opening monologue introducing himself, Paperspace, and GPU infrastructure. Host Matt Turck does not participate in this segment. | |
| Differentiating Pipeline Primitives from Algorithmic Primitives | 0 | 1 | 0 | 0 | Dillon presents a solo talk comparing current home-rolled ML pipelines to 1990s web development. The host is absent during this monologue segment. | |
| Infrastructure and Tooling as the Primary Bottleneck to AI Adoption | 0 | 1 | 0 | 0 | Dillon argues that tooling and infrastructure, rather than algorithm design, form the main barrier to AI adoption. The host does not participate. | |
| Deconstructing Traditional Web Application CI/CD Workflows | 0 | 1 | 0 | 0 | Dillon contrasts traditional linear CI/CD code workflows with data-driven ML pipelines requiring specialized hardware. The host is not involved in this monologue. | |
| Defining New Semantics, Triggers, and Abstractions for ML | 0 | 1 | 0 | 0 | Dillon outlines new ML pipeline triggers such as data drift and model drift. The host remains silent during this monologue segment. | |
| Presentation Wrap-Up and Researcher-in-Residence Announcement | 4 | 2 | 1 | 3 | Host Matt Turck kicks off Q&A by pressing Dillon on how a startup can defend against major cloud providers expanding their AI offerings. Dillon candidly admits it is a hard question before explaining Paperspace's software-abstraction positioning. |
Statements from this episode (14)
Erb: Existing CI/CD workflows do not fit machine learning pipelines
“Existing CI CD workflows, talking about how those don't really fit anymore. And then making the case that you know, these new pipelines are fundamentally different and require new tools, new workflows, and generally like new paradigms.”
Erb: AI researchers and infrastructure teams view algorithmic primitives completely differently
“So if you're an AI or machine learning or deep learning developer or researcher, you think of your primitives as, you know, hidden layers and LSTMs and things like that. From an infrastructure perspective, it's actually much higher up”
Tech giants like Google and Uber rely on home-rolled ML infrastructure
“Most of the tooling is kind of home rolled. Best practices haven't emerged yet. We're seeing a lot of, especially large companies, roll their entire own stacks. So Facebook has FB Learner, Google has TFX. Uber has Michelangelo. Airbnb has Big Head.”
Erb: Billions have been invested in pre-product AI hardware unicorns
“A few billion dollars have been invested in new hardware startups. There are a few unicorns already for companies that don't have products fully out yet.”
Infrastructure, not algorithms, is the primary barrier to AI adoption
“I'm gonna make the case that the biggest barrier to adoption is an infrastructure and tooling problem, not necessarily an algorithmic problem.”
CI/CD principles address the black-box interpretability problem in deep learning
“You care about it because it adds reliability, reproducibility, determinism. You know, your systems are no longer inscrutable, which is a very big criticism, especially of deep learning architectures, which is that they're black box or they're, you know, they …”
Erb predicts ten new machine learning hardware accelerators will launch in 2019
“My guess is by the end of the year, there will be 10 more new devices that are coming out.”
Paperspace customers run machine learning training workloads lasting up to three weeks
“We run workloads for customers that can go up to like three weeks for a training task”
Machine learning systems must trigger pipelines based on data or model drift
“Code drift there, which arguably is the number one, is the only trigger for most web apps. You know, your code drift, someone adds something, you rebuild your model, which is your web application. But actually what you care about in these new systems is data d…”
Erb: AI ops tools will expand into broad platforms within two years
“Tools in this space, this is an observation, ah, I'll check back in in a year or two, we'll see if I was totally wrong. Will go wide and deep to close this AI ops feedback loop.”
Software platforms will almost entirely abstract away underlying machine learning hardware
“Hardware will also be almost entirely abstracted away by these platforms, which is going to be an enormous mountain of work that will be done by lots of companies.”
Erb: The largest deep learning teams generally do not exceed 20 people
“The biggest teams generally don't go much larger than, say, 15 to 20 people, especially in the deep learning space.”
Lack of standard model repositories forces companies to hack solutions using Dropbox
“There's no good model zoo or model repo, and, you know, almost every company that I've worked with has their own version of that, and it can range from a Dropbox folder to, you know, actually repurposing Travis to actually do that.”
Deep learning teams using standard code branches are light years ahead
“If you find, you know, companies that are doing deep learning that have, you know, staging development and master branches, that's already, like, light years ahead of, I would say, 99% of the companies that will be using this technology in five years.”