SAM-II

“And we find that actually SAM-II is a lot better than SAM when it comes to segmenting objects in video frames, because they actually have a sort of slightly different distribution than images.”

“We started with just SAM. We apply SAM per frame. That's like the most basic way of extending SAM to video. Then the most obvious thing to do is to take the output masks from SAM and then provide it as input into a video object segmentation model that takes th…”

“There's like underwater videos that it works actually really well for, even though we, models never really seen an octopus before. And octopus have a lot of Moving parts that SAM-II can actually quite effectively keep track of all the different tentacles.”

“Essentially, idea of a concept prompt opens up the ability to find all instances of an object category without having to manually click on every single instance, as you would have had to do if you were using SAM-II or SAM-I.”