Zhang: SA-Co Benchmark Will Likely Outlast SAM 3
Pengchuan Zhang · SAM 3: The Eyes for AI — Nikhila & Pengchuan (Meta Superintelligence), ft. Joseph Nelson (Roboflow) · Dec 18, 2025 · at 1:13:23
Pengchuan Zhang, AI researcher at Meta Superintelligence / FAIR, discusses the release of SAM 3 and the long-term utility of the accompanying benchmark for tracking AI vision capabilities.
0:00 / 0:19exact quote · 19.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I would say that it's likely that the benchmark will last longer than our Samsung model. Maybe kind of next year there will be a stronger model, but the benchmark is kind of the one that I hope to guide the community to kind of get better and better models kind of to get to a kind of way measure human performance on the benchmark.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Pengchuan Zhang
Zhang: Superhuman computer vision requires RLHF rather than human SFT data
“But if you only do SFT and the SFT data is annotated by human, then your performance is funded by human. You cannot get, kind of, superhuman performance just by, kind of, this kind of data engine approach to use human annotated data and then learn from that. Y…”
Zhang: AI models will handle simple vision natively, using tools for complexity
“I think at least I want to bet on, you know, running their work natively together, the future for simple, I would say for simple or even intermediate difficult vision tasks. For example, kind of counting with less than 20 objects. I think for this kind of simp…”
Zhang: Fine-Tuned Llama 3.2 Achieved Superhuman Vision Verification Performance
“We kind of fine-tune our, kind of, for example, NAMA's 3.2 with our, kind of, verification, human annotated verification data. We get, kind of, superhuman performance on these two verification tasks, and then we do not need human on these two tasks. Let's furt…”
Zhang: SAM 3 achieves real-time tracking across objects via multi-GPU parallelism
“Even for video, if you can't afford the kind of GPUs, pretty many, very kind of, do the kind of parallel inference algorithm. So even you have a lot of object to track, you can still get real-time tracking performance as long as you scale up the GPUs there.”