Grounded SAM：组装开放世界模型以完成多样化视觉任务
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
AI总结：
本文提出 Grounded SAM，将 Grounding DINO 与 SAM 及多种视觉模型组合，实现文本驱动的开放世界检测分割、自动标注、图像编辑和 3D 运动分析，并在 SegInW 零样本基准达到 48.7 mAP。
摘要：
我们介绍了 Grounded SAM，它使用 Grounding DINO 作为开放集目标检测器，并与 segment anything model（SAM）相结合。这种集成使得基于任意文本输入检测并分割任何区域成为可能，并为连接各种视觉模型打开了大门。如图 1 所示，通过使用多功能的 Grounded SAM 流程，可以完成广泛的视觉任务。例如，仅基于输入图像的自动标注流程可以通过结合 BLIP 和 Recognize Anything 等模型来实现。此外，结合 Stable-Diffusion 可实现可控图像编辑，而集成 OSX 则有助于可提示的 3D 人体运动分析。Grounded SAM 在开放词汇基准测试中也表现出优越性能，在 Grounding DINO-Base 与 SAM-Huge 模型组合下，于 SegInW（Segmentation in the wild）零样本基准测试上取得 48.7 平均 AP。
英文摘要：
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM). This integration enables the detection and segmentation of any regions based on arbitrary text inputs and opens a door to connecting various vision models. As shown in Fig.1, a wide range of vision tasks can be achieved by using the versatile Grounded SAM pipeline. For example, an automatic annotation pipeline based solely on input images can be realized by incorporating models such as BLIP and Recognize Anything. Additionally, incorporating Stable-Diffusion allows for controllable image editing, while the integration of OSX facilitates promptable 3D human motion analysis. Grounded SAM also shows superior performance on open-vocabulary benchmarks, achieving 48.7 mean AP on SegInW (Segmentation in the wild) zero-shot benchmark with the combination of Grounding DINO-Base and SAM-Huge models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
