Nelson: Smallest SAM 2 model has 38M parameters and runs at 45 FPS
Joseph Nelson · Segment Anything 2: Memory + Vision = Object Permanence — with Nikhila Ravi and Joseph Nelson · Aug 7, 2024 · at 52:54
Joseph Nelson highlights the compact size and inference speed of Meta FAIR's Segment Anything 2 (SAM 2) model family.
0:00 / 0:04exact quote · 4.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“The smallest model is thirty-eight million parameters and can run at 45 FPS on an A-one hundred, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Joseph Nelson
Nelson: Users Ran 8 Million Inferences in SAM 3's First Five Days
“I mean, in the first five days of SAM three, there was like eight million inferences of folks that were running across all diverse sets of fields.”
Nelson: A Single Negative Example Goes a Long Way in Vision Fine-Tuning
“I can offer anecdotally that a single negative example goes a long way.”
Nelson: SAM 3 Open Text Boxes Will Trigger Many Unprimed User Queries
“Now that you've kind of given this open text box for media, there's going to be a flood of the types of things users are going to want to try to do, some of which SAM is already going to be really well adapted to do, some of which not.”
Nelson: Last-mile computer vision requires aligning intention, not knowledge
“And so this is why, like, in some ways human in the loop, because identifying human intention, not necessarily human knowledge is what's going to be important for a lot of last mile use.”
Production computer vision still requires user prompting to isolate targets
“But even if you had, like, a perfect SAM, like an omniscient SAM that could see every segment in every domain with all pixels perfectly outlined, in production, you still need some way to almost, like, signal to the model what you care about.”