Meta FAIR

“And we find that actually SAM-II is a lot better than SAM when it comes to segmenting objects in video frames, because they actually have a sort of slightly different distribution than images.”

“We started with just SAM. We apply SAM per frame. That's like the most basic way of extending SAM to video. Then the most obvious thing to do is to take the output masks from SAM and then provide it as input into a video object segmentation model that takes th…”

“And in terms of the efficiency compared to SAM, so if we were to run SAM per frame on a video or run SAM two, it's around six times faster to run SAM two versus run SAM per frame.”

“So as you've probably seen with SAM and SAM-II, it's a fairly narrow problem, but we really try to make it a step change in the capability. And so with each Version. We are trying to limit the focus on one thing that we can know we can do really well. And in t…”

“And the other piece here is that the demo is actually the annotation tool. So we actually Use the demo as a way to improve our annotation tool. And so then it becomes very natural to invest in building a good demo because it speeds up your annotation and impro…”

“Combining two models and sort of just smushing things together might not actually be as effective as if you really think about how to build things in a unified way.”

“SAM-I model was around six hundred and thirty million parameters, a fraction of the size of these large language models, but very small. Actually SAM-II, the largest model is around two hundred and twenty-four million parameters. There's actually One third the…”

“Each of the, it scales with the number of detected objects.”

“The detector needs to be identity agnostic. So if you have a concept dog, it needs to be able to find all instances of that dog. And it needs to sort of have this representation of dog that is the same for all dogs. But when you're tracking those dogs through …”

“We are really having a unified model that can do many different tasks in the same unified architecture. And so, you know, then the same way that LLMs can do many different tasks without needing a task-specific model. Like with SAM-III, we're able to do image-p…”

“And so my team, when we were at MetaFair, trained really the first transformer language model for protein biology.”