May 28, 2015 · 24m · mad
David Luan, Dextro // Real-World Video Understanding (FirstMark / Data Driven NYC)
David Luan · 18m spoken Matt Turck · 1m spoken Andrew / Eric Smith · 19s spoken Richard Blue · 9s spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
At Data Driven NYC, David Luan presents Dextro's computer vision and deep learning platform, explaining how automated feature extraction, timeline analysis, live stream processing, and custom taxonomy mapping convert unstructured video streams into actionable metadata for media enterprises.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 5.6% of the talking time here. How this is scored →
Matt as informed peer 0.7 Guest teaching 3.0 Guest disagreement 0.2 Matt pushing back 0.3
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 18:12 Deflecting Unreleased Feature Details
When asked if audio cues like truck crash sounds are integrated into the model, David humorously deflects the question with 'Let's talk in a couple months', setting a firm boundary on unannounced technical features.
Hardest push from Matt ▶ 15:18 Challenging the Vertical AI PremiseMatt Turck pushes past the guest's thesis on vertical focus by asking whether a universal horizontal AI layer is coming or remains pure science fiction.
Biggest teaching moment ▶ 16:10 Distinguishing Algorithms from Last-Mile SolutionsDavid educates the host and audience on why raw deep learning algorithms fail commercially without the additional last-mile labor of data collection and fine-tuning for specific customer workflows.
Matt holds his own ▶ 14:31 Proposing Real-Time Predictive Video AnalyticsMatt Turck demonstrates keen technical foresight by asking whether real-time computer vision can evolve into spatial-temporal predictive analytics for tracking live movement.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Dextro's Core Mission and Scale of Video Data | 0 | 2 | 0 | 0 | Presentation monologue where David Luan introduces Dextro and outlines the vast scale of video uploaded to platforms like YouTube. The host does not speak in this segment. | |
| Developer-Friendly JSON API Infrastructure | 0 | 2 | 0 | 0 | David continues his presentation on Dextro's JSON API output and real-time live streaming analysis on Periscope. Host is silent during the presentation. | |
| High-Value Applications: Discovery, Curation, and Audience Insights | 0 | 3 | 0 | 0 | David outlines historical milestones in deep learning, such as Krzyzewski's ImageNet neural network and Karpathy's image captioning. Monologue format with no host interaction. | |
| The Allure of Doing Everything vs. Vertical Focus | 0 | 4 | 0 | 0 | David explains the strategic trade-off between building general horizontal ML tools and solving vertical customer problems. Uninterrupted presentation. | |
| Real-World Content vs. Iconic Stock Images | 0 | 3 | 0 | 0 | David contrasts ideal iconic stock photos with noisy real-world video content and explains how Dextro handles partner taxonomies. Monologue format. | |
| Incorporating Motion Cues with Video-Specific Models | 4 | 4 | 1 | 2 | Matt Turck opens Q&A with conceptual questions regarding predictive video analytics and horizontal AI layers, prompting David and audience members to discuss practical video AI applications. |
Statements from this episode (14)
In 2015, 300 hours of video were uploaded to YouTube every minute
“Around 300 hours of video, it's probably even more now actually, are uploaded to YouTube every minute right now.”
Dextro uses a salience graph to measure video concept prominence
“So what we do is we provide what's, what's also a salience graph, which is a discounted score of how important every concept Or how prominent a particular category is over the video as a whole. So it's not a measure of our confidence, but of actually how impor…”
Dextro was the first to offer automated video analysis as a service
“We were the first company to figure out how to get this level of analysis of what's happening in videos as a service.”
Dextro analyzes video strictly through computer vision, not metadata
“This is all just done with computer vision. We don't use any of the metadata whatsoever to identify what's actually happening.”
AlexNet halved state-of-the-art visual recognition error rates in a single year
“Krzyzewski's work on ILS VRC, which is the ImageNet Large Scale Visual Recognition Challenge, where in one year, essentially they blew away the previous state of the art with a deep convolutional neural network by about, like, half of the final error rate on t…”
Academic reviewers are tired of papers blindly applying deep learning
“And I think the reviewers now are pretty tired of that and they're moving on, but.”
Enterprise customers will not pay for 80% accurate machine learning
“But in most cases, customers aren't willing to pay for a product that only gets them 80% of the way. You have to kind of like specialize and focus on the problem to make sure that you get to being 100%.”
Models trained on stock images fail to generalize to real-world video
“Classifiers that are and models that are trained in tune on this particular, on iconic type of data don't really generalize as well when you apply it to something that you might see in a real-world video like on YouTube or on Periscope.”
Frame-level tags lack the high-level semantic context required for video discovery
“Video or frame-level tags, the sort that you might see on, that I've put on the screen right now, didn't actually solve their problem. Because that was, it was too low level in like a, in a, in not in terms of a granularity sense, but in terms of how much addi…”
Dextro built architecture to map custom customer taxonomies without retraining models
“Our core machine learning system needed to be able to easily adapt and generalize to customer and partner taxonomies without restarting training and data collection and everything like that from scratch every single time. And so how we solved that problem was …”
Frame-by-frame video analysis discards critical temporal motion data
“The naive approach to generalizing the video is to analyze videos as just a frame by frame sequence of photos. But there's so much encoded in the motion information in a video that to just do that actually just throws all of it out.”
Enterprise deep learning will remain vertical before converging horizontally
“In terms of where we're going to see it In industry, it's likely to be still in a kind of vertical by vertical system for a little while before we start seeing a little more convergence.”
Public user-generated video tags are too noisy for training vision models
“So with regard to using tags that are already present on the internet, we run into a bunch of different problems, which is that, one, so there is some information to be gained there in general, but it's extremely noisy, and the, another big thing that we see i…”
Dextro created a real-time aggregator for all live public Periscope streams
“Tomorrow we're just ironing out that one issue, but you guys should all check out stream.dextro.co, which aggregates every live public Periscope stream, and you can kind of browse based on what you think is most interesting at any given moment to discover, lik…”