Everything Dillon Erb said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Erb: Existing CI/CD workflows do not fit machine learning pipelines
“Existing CI CD workflows, talking about how those don't really fit anymore. And then making the case that you know, these new pipelines are fundamentally different and require new tools, new workflows, and generally like new paradigms.”
Erb: Billions have been invested in pre-product AI hardware unicorns
“A few billion dollars have been invested in new hardware startups. There are a few unicorns already for companies that don't have products fully out yet.”
Infrastructure, not algorithms, is the primary barrier to AI adoption
“I'm gonna make the case that the biggest barrier to adoption is an infrastructure and tooling problem, not necessarily an algorithmic problem.”
Erb: AI researchers and infrastructure teams view algorithmic primitives completely differently
“So if you're an AI or machine learning or deep learning developer or researcher, you think of your primitives as, you know, hidden layers and LSTMs and things like that. From an infrastructure perspective, it's actually much higher up”
Tech giants like Google and Uber rely on home-rolled ML infrastructure
“Most of the tooling is kind of home rolled. Best practices haven't emerged yet. We're seeing a lot of, especially large companies, roll their entire own stacks. So Facebook has FB Learner, Google has TFX. Uber has Michelangelo. Airbnb has Big Head.”
CI/CD principles address the black-box interpretability problem in deep learning
“You care about it because it adds reliability, reproducibility, determinism. You know, your systems are no longer inscrutable, which is a very big criticism, especially of deep learning architectures, which is that they're black box or they're, you know, they …”
Erb predicts ten new machine learning hardware accelerators will launch in 2019
“My guess is by the end of the year, there will be 10 more new devices that are coming out.”
Machine learning systems must trigger pipelines based on data or model drift
“Code drift there, which arguably is the number one, is the only trigger for most web apps. You know, your code drift, someone adds something, you rebuild your model, which is your web application. But actually what you care about in these new systems is data d…”
Erb: AI ops tools will expand into broad platforms within two years
“Tools in this space, this is an observation, ah, I'll check back in in a year or two, we'll see if I was totally wrong. Will go wide and deep to close this AI ops feedback loop.”
Software platforms will almost entirely abstract away underlying machine learning hardware
“Hardware will also be almost entirely abstracted away by these platforms, which is going to be an enormous mountain of work that will be done by lots of companies.”
Erb: The largest deep learning teams generally do not exceed 20 people
“The biggest teams generally don't go much larger than, say, 15 to 20 people, especially in the deep learning space.”
Lack of standard model repositories forces companies to hack solutions using Dropbox
“There's no good model zoo or model repo, and, you know, almost every company that I've worked with has their own version of that, and it can range from a Dropbox folder to, you know, actually repurposing Travis to actually do that.”
Deep learning teams using standard code branches are light years ahead
“If you find, you know, companies that are doing deep learning that have, you know, staging development and master branches, that's already, like, light years ahead of, I would say, 99% of the companies that will be using this technology in five years.”
Paperspace customers run machine learning training workloads lasting up to three weeks
“We run workloads for customers that can go up to like three weeks for a training task”