X2SAM：图像和视频中的任意分割
X2SAM: Any Segmentation in Images and Videos
AI总结：
X2SAM是一种统一的分割多模态大语言模型，能够将图像分割能力扩展到视频，结合LLM和Mask Memory模块，支持通用、开放词汇、指称、推理、基于视觉的对话生成及交互式分割。
摘要：
多模态大语言模型（MLLMs）在图像层面的视觉理解和推理方面表现出色，但其在图像和视频层面的像素级感知仍有限。基础分割模型如SAM系列能生成高质量的掩码，但依赖低级视觉提示且无法原生解释复杂的对话指令。现有分割MLLMs缩小了这一差距，但通常专门针对图像或视频，很少支持文本和视觉提示的统一接口。我们引入X2SAM，一种统一的分割MLLM，将任意分割能力从图像扩展到视频。给定对话指令和视觉提示，X2SAM结合LLM和Mask Memory模块，存储引导的视觉特征以生成时间一致的视频掩码生成。相同的公式支持通用、开放词汇、指称、推理、基于视觉的对话生成、交互式和基于视觉的分割，适用于图像和视频输入。我们进一步引入视频视觉基于（V-VGD）分割基准，评估模型是否能从交互式视觉提示中分割视频中的物体轨迹。通过在异构图像和视频数据集上的统一联合训练策略，X2SAM在视频分割任务中表现强劲，在图像分割基准中保持竞争力，并保留通用图像和视频聊天能力。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated strong image-level visual understanding and reasoning, yet their pixel-level perception across both images and videos remains limited. Foundation segmentation models such as the SAM series produce high-quality masks, but they rely on low-level visual prompts and cannot natively interpret complex conversational instructions. Existing segmentation MLLMs narrow this gap, but are usually specialized for either images or videos and rarely support both textual and visual prompts in one interface. We introduce X2SAM, a unified segmentation MLLM that extends any-segmentation capabilities from images to videos. Given conversational instructions and visual prompts, X2SAM couples an LLM with a Mask Memory module that stores guided vision features for temporally consistent video mask generation. The same formulation supports generic, open-vocabulary, referring, reasoning, grounded conversation generation, interactive, and visual grounded segmentation across image and video inputs. We further introduce the Video Visual Grounded (V-VGD) segmentation benchmark, which evaluates whether a model can segment object tracks in videos from interactive visual prompts. With a unified joint training strategy over heterogeneous image and video datasets, X2SAM delivers strong video segmentation performance, remains competitive on image segmentation benchmarks, and preserves general image and video chat ability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
