Frame-by-frame video analysis discards critical temporal motion data
David Luan · David Luan, Dextro // Real-World Video Understanding (FirstMark / Data Driven NYC) · May 28, 2015 · at 13:02
David Luan, co-founder of video analytics startup Dextro, explains why naive computer vision methods fail on video content during a talk at Data Driven NYC.
0:00 / 0:12exact quote · 12.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The naive approach to generalizing the video is to analyze videos as just a frame by frame sequence of photos. But there's so much encoded in the motion information in a video that to just do that actually just throws all of it out.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from David Luan
Dextro was the first to offer automated video analysis as a service
“We were the first company to figure out how to get this level of analysis of what's happening in videos as a service.”
Academic reviewers are tired of papers blindly applying deep learning
“And I think the reviewers now are pretty tired of that and they're moving on, but.”
Enterprise customers will not pay for 80% accurate machine learning
“But in most cases, customers aren't willing to pay for a product that only gets them 80% of the way. You have to kind of like specialize and focus on the problem to make sure that you get to being 100%.”
Dextro analyzes video strictly through computer vision, not metadata
“This is all just done with computer vision. We don't use any of the metadata whatsoever to identify what's actually happening.”
Models trained on stock images fail to generalize to real-world video
“Classifiers that are and models that are trained in tune on this particular, on iconic type of data don't really generalize as well when you apply it to something that you might see in a real-world video like on YouTube or on Periscope.”
Frame-level tags lack the high-level semantic context required for video discovery
“Video or frame-level tags, the sort that you might see on, that I've put on the screen right now, didn't actually solve their problem. Because that was, it was too low level in like a, in a, in not in terms of a granularity sense, but in terms of how much addi…”