StarZero All shows Try the podcast clipper →

Mar 2, 2018 · 24m · mad

Building an Operating System for AI // Diego Oppenheimer, Algorithmia (FirstMark's Data Driven)

Diego Oppenheimer · 18m spoken Matt Turck · 37s spoken
0:00 / 0:00
▶ Watch on YouTube →

gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions

At FirstMark's DataDrivenNYC, Algorithmia CEO Diego Oppenheimer presents a blueprint for an Operating System for AI designed to solve machine learning inference and scaling challenges. He explains how serverless microservice architecture dramatically slashes compute costs and enables seamless multi-cloud model deployment for enterprises.

How this conversation actually went

Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 3.2% of the talking time here. How this is scored →

Matt as informed peer 0.9 Guest teaching 4.9 Guest disagreement 0.3 Matt pushing back 0.4
05100:0010:0020:001:22–5:04 · Matt as informed peer 0/10 Algorithmia Platform & Scale Overview Diego presents a monologue outlining Algorithmia's platform scale and the distinction between model training and inference. Because this is a presentation monologue, host scores are strictly zero.5:04–9:49 · Matt as informed peer 0/10 The 'SeeFood' App Scaling Dilemma Diego uses the 'SeeFood' hot dog app example to explain why serverless microservices fit AI inference demands. Host participation is zero during this presentation section.9:49–12:19 · Matt as informed peer 0/10 Architecture Cost & Efficiency Analysis Diego breaks down traditional GPU capacity planning waste compared to serverless auto-scaling cost benefits. As a monologue segment, host metrics remain zero.12:19–16:23 · Matt as informed peer 0/10 Core Components of an AI Operating System Diego details the architecture of an AI operating system, focusing on kernel execution and model composability. Host metrics are zero during this solo monologue.16:23–20:00 · Matt as informed peer 2/10 Presentation Conclusion & Promo Offer Matt asks how Diego sells to regulated industries and expresses doubt about cloud adoption there, prompting Diego to explain their on-premise firewall model. An audience member asks how they compete with Amazon SageMaker and IBM.20:00–22:38 · Matt as informed peer 3/10 Q&A: Marketplace Benchmarking & Company History Diego responds to an audience question on algorithm benchmarking before Matt asks about company history. Matt shows domain knowledge by supplying the name of Google's AI fund, Gradient, before Diego finishes his phrase.22:38–24:05 · Matt as informed peer 1/10 Q&A: Cloud Partnerships & Event Conclusion Diego answers an audience question regarding cloud provider relationships and revenue dynamics. Matt concludes the presentation and smoothly transitions to networking.1:22–5:04 · Guest teaching 5/10 Algorithmia Platform & Scale Overview Diego presents a monologue outlining Algorithmia's platform scale and the distinction between model training and inference. Because this is a presentation monologue, host scores are strictly zero.5:04–9:49 · Guest teaching 5/10 The 'SeeFood' App Scaling Dilemma Diego uses the 'SeeFood' hot dog app example to explain why serverless microservices fit AI inference demands. Host participation is zero during this presentation section.9:49–12:19 · Guest teaching 6/10 Architecture Cost & Efficiency Analysis Diego breaks down traditional GPU capacity planning waste compared to serverless auto-scaling cost benefits. As a monologue segment, host metrics remain zero.12:19–16:23 · Guest teaching 6/10 Core Components of an AI Operating System Diego details the architecture of an AI operating system, focusing on kernel execution and model composability. Host metrics are zero during this solo monologue.16:23–20:00 · Guest teaching 5/10 Presentation Conclusion & Promo Offer Matt asks how Diego sells to regulated industries and expresses doubt about cloud adoption there, prompting Diego to explain their on-premise firewall model. An audience member asks how they compete with Amazon SageMaker and IBM.20:00–22:38 · Guest teaching 4/10 Q&A: Marketplace Benchmarking & Company History Diego responds to an audience question on algorithm benchmarking before Matt asks about company history. Matt shows domain knowledge by supplying the name of Google's AI fund, Gradient, before Diego finishes his phrase.22:38–24:05 · Guest teaching 3/10 Q&A: Cloud Partnerships & Event Conclusion Diego answers an audience question regarding cloud provider relationships and revenue dynamics. Matt concludes the presentation and smoothly transitions to networking.1:22–5:04 · Guest disagreement 0/10 Algorithmia Platform & Scale Overview Diego presents a monologue outlining Algorithmia's platform scale and the distinction between model training and inference. Because this is a presentation monologue, host scores are strictly zero.5:04–9:49 · Guest disagreement 0/10 The 'SeeFood' App Scaling Dilemma Diego uses the 'SeeFood' hot dog app example to explain why serverless microservices fit AI inference demands. Host participation is zero during this presentation section.9:49–12:19 · Guest disagreement 0/10 Architecture Cost & Efficiency Analysis Diego breaks down traditional GPU capacity planning waste compared to serverless auto-scaling cost benefits. As a monologue segment, host metrics remain zero.12:19–16:23 · Guest disagreement 0/10 Core Components of an AI Operating System Diego details the architecture of an AI operating system, focusing on kernel execution and model composability. Host metrics are zero during this solo monologue.16:23–20:00 · Guest disagreement 1/10 Presentation Conclusion & Promo Offer Matt asks how Diego sells to regulated industries and expresses doubt about cloud adoption there, prompting Diego to explain their on-premise firewall model. An audience member asks how they compete with Amazon SageMaker and IBM.20:00–22:38 · Guest disagreement 1/10 Q&A: Marketplace Benchmarking & Company History Diego responds to an audience question on algorithm benchmarking before Matt asks about company history. Matt shows domain knowledge by supplying the name of Google's AI fund, Gradient, before Diego finishes his phrase.22:38–24:05 · Guest disagreement 0/10 Q&A: Cloud Partnerships & Event Conclusion Diego answers an audience question regarding cloud provider relationships and revenue dynamics. Matt concludes the presentation and smoothly transitions to networking.1:22–5:04 · Matt pushing back 0/10 Algorithmia Platform & Scale Overview Diego presents a monologue outlining Algorithmia's platform scale and the distinction between model training and inference. Because this is a presentation monologue, host scores are strictly zero.5:04–9:49 · Matt pushing back 0/10 The 'SeeFood' App Scaling Dilemma Diego uses the 'SeeFood' hot dog app example to explain why serverless microservices fit AI inference demands. Host participation is zero during this presentation section.9:49–12:19 · Matt pushing back 0/10 Architecture Cost & Efficiency Analysis Diego breaks down traditional GPU capacity planning waste compared to serverless auto-scaling cost benefits. As a monologue segment, host metrics remain zero.12:19–16:23 · Matt pushing back 0/10 Core Components of an AI Operating System Diego details the architecture of an AI operating system, focusing on kernel execution and model composability. Host metrics are zero during this solo monologue.16:23–20:00 · Matt pushing back 2/10 Presentation Conclusion & Promo Offer Matt asks how Diego sells to regulated industries and expresses doubt about cloud adoption there, prompting Diego to explain their on-premise firewall model. An audience member asks how they compete with Amazon SageMaker and IBM.20:00–22:38 · Matt pushing back 1/10 Q&A: Marketplace Benchmarking & Company History Diego responds to an audience question on algorithm benchmarking before Matt asks about company history. Matt shows domain knowledge by supplying the name of Google's AI fund, Gradient, before Diego finishes his phrase.22:38–24:05 · Matt pushing back 0/10 Q&A: Cloud Partnerships & Event Conclusion Diego answers an audience question regarding cloud provider relationships and revenue dynamics. Matt concludes the presentation and smoothly transitions to networking.

speaking balance: gold is Matt, purple is the guest (3 minute bins)

0:00 · Matt 0% · guest 100%0:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%3:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%6:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%9:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%12:00 · Matt 0% · guest 100%15:00 · Matt 8.3% · guest 91.7%15:00 · Matt 8.3% · guest 91.7%18:00 · Matt 0.5% · guest 99.5%18:00 · Matt 0.5% · guest 99.5%21:00 · Matt 15.9% · guest 84.1%21:00 · Matt 15.9% · guest 84.1%24:00 · Matt 100% · guest 0%24:00 · Matt 100% · guest 0%
Sharpest disagreement ▶ 18:36 Reframing Training vs Inference Separation

Diego politely rejects the audience member's premise that Algorithmia divorces training from inference, explaining that production machine learning operates as a continuous CI/CD pipeline.

 Hardest push from Matt ▶ 17:19 Questioning Enterprise Cloud Adoption

Matt pushes back on Diego's target customer list by questioning whether regulated enterprise clients would accept a cloud solution, prompting Diego to clarify their on-premise firewall deployment model.

 Biggest teaching moment ▶ 10:30 Demonstrating Serverless Cost Math

Diego educates the audience on traditional GPU capacity planning inefficiencies compared to serverless auto-scaling, demonstrating how load-balancing models achieve up to 85 percent cost savings.

 Matt holds his own ▶ 21:59 Identifying Google's AI Fund

Matt displays immediate domain awareness by interrupting to name 'Gradient' as Google's dedicated AI investment fund before Diego finishes naming it.
the scores for every segment, with the reasoning behind each
ChapterTopicMatt as informed peerGuest teachingGuest disagreementMatt pushing backWhy
Algorithmia Platform & Scale Overview 0500 Diego presents a monologue outlining Algorithmia's platform scale and the distinction between model training and inference. Because this is a presentation monologue, host scores are strictly zero.
The 'SeeFood' App Scaling Dilemma 0500 Diego uses the 'SeeFood' hot dog app example to explain why serverless microservices fit AI inference demands. Host participation is zero during this presentation section.
Architecture Cost & Efficiency Analysis 0600 Diego breaks down traditional GPU capacity planning waste compared to serverless auto-scaling cost benefits. As a monologue segment, host metrics remain zero.
Core Components of an AI Operating System 0600 Diego details the architecture of an AI operating system, focusing on kernel execution and model composability. Host metrics are zero during this solo monologue.
Presentation Conclusion & Promo Offer 2512 Matt asks how Diego sells to regulated industries and expresses doubt about cloud adoption there, prompting Diego to explain their on-premise firewall model. An audience member asks how they compete with Amazon SageMaker and IBM.
Q&A: Marketplace Benchmarking & Company History 3411 Diego responds to an audience question on algorithm benchmarking before Matt asks about company history. Matt shows domain knowledge by supplying the name of Google's AI fund, Gradient, before Diego finishes his phrase.
Q&A: Cloud Partnerships & Event Conclusion 1300 Diego answers an audience question regarding cloud provider relationships and revenue dynamics. Matt concludes the presentation and smoothly transitions to networking.

Statements from this episode (12)

Assertion Not checkable as stated
Diego Oppenheimer states Algorithmia has 63,000 developers using its platform
“We have 63,000 developers that use our platform.”
Diego Oppenheimer Mar 2, 2018 ▶ 2:09
Assertion Not checkable as stated
Diego Oppenheimer claims Algorithmia manages over 50,000 functions simultaneously
“We're actually dealing with over 50,000 different functions at any given time that need to be able to deploy it and run on the infrastructure.”
Diego Oppenheimer Mar 2, 2018 ▶ 2:33
Assertion Not checkable as stated
Diego Oppenheimer states Algorithmia operates under 15 milliseconds of latency overhead
“We sit to a strict latency of under 15 milliseconds of overhead throughout an entire system.”
Diego Oppenheimer Mar 2, 2018 ▶ 2:51
Insight
Oppenheimer: TensorFlow is open source, but scaling it is not
“TensorFlow is open source, but actually scaling it is not.”
Diego Oppenheimer Mar 2, 2018 ▶ 3:48
Opinion
Oppenheimer: AI and machine learning are the killer apps for serverless
“I actually think that AI and machine learning are the killer apps for serverless microservices.”
Diego Oppenheimer Mar 2, 2018 ▶ 6:39
Insight
Oppenheimer: ML inference belongs to DevOps, not data scientists
“We think that the inference side of things is actually owned by DevOps and the infrastructure teams”
Diego Oppenheimer Mar 2, 2018 ▶ 6:51
Assertion Not checkable as stated
Oppenheimer: Serverless AI architecture can cut infrastructure costs by up to 85%
“Using serverless, we can actually track as the demand comes in. We can use our load balancers to quickly initiate and replicate these models over the entire infrastructure, going up and down with the demand, allowing for savings of up to 85% of the infrastruct…”
Diego Oppenheimer Mar 2, 2018 ▶ 10:33
Insight
Oppenheimer: Data is heavy and expensive to move, but compute is light and cheap
“So there's a design principle that we use, personally, at Algorithmia, which is data is heavy and expensive to move, but compute is light and cheap.”
Diego Oppenheimer Mar 2, 2018 ▶ 11:40
Assertion Supported
Oppenheimer: Algorithmia is likely the only serverless GPU provider
“We're probably the only product out there that does serverless over GPUs, and it's really important from a price and a performance perspective, but all the thing is that we actually do cross-cloud.”
Diego Oppenheimer Mar 2, 2018 ▶ 19:12
Disclosure
Algorithmia has raised about $13.5M with a team of 24 people
“So so we're based out of Seattle, headquartered in Seattle, have a small office here in New York. Team of 24 people at this point raised about 13 and a half million dollars. Our latest round was done by Google's AI fund.”
Diego Oppenheimer Mar 2, 2018 ▶ 21:42
Assertion Not checkable as stated
Oppenheimer: Cloud providers earn over $1 per dollar Algorithmia makes
“For every dollar that we you know, that we make, the infrastructure provider under us is making more than that dollar.”
Diego Oppenheimer Mar 2, 2018 ▶ 22:42
Disclosure
Oppenheimer: Algorithmia co-sells products directly with AWS and Azure
“Some of them we actually do co-selling experiences with them where, you know, we're, you know, we're informing their sales forces to say, hey, this is a product that exists on AWS, this is a product that exists on Azure, And you can actually go ahead and use i…”
Diego Oppenheimer Mar 2, 2018 ▶ 23:15
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 400 conversations transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more