UniPixel：面向像素级视觉推理的统一对象指代与分割
UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning
AI总结：
UniPixel提出统一指代与分割的大模型，通过视觉提示生成掩码并以此为条件进行推理，在10个基准及新设计的PixelQA任务上验证了像素级视觉推理能力。
摘要：
大型多模态模型（LMMs）的最新进展已证明其作为通用多模态助手的显著成功，尤其侧重于整体图像和视频语言理解。相比之下，对扩展细粒度像素级理解能力的关注较少，该能力要求模型实现视觉信号与语言语义之间的像素级对齐。一些先前研究已将LMMs应用于区域级描述和指代表达分割等相关任务。然而，这些模型仅限于独立执行指代或分割任务，未能将这些细粒度感知能力整合到视觉推理中。为弥合这一差距，我们提出UniPixel，这是一种能够灵活理解视觉提示输入并生成以掩码为锚定响应的大型多模态模型。我们的模型通过将像素级感知与通用视觉理解能力无缝集成而独树一帜。具体而言，UniPixel处理视觉提示并按需生成相关掩码，在推理过程中以这些中间指针为条件执行后续推理，从而实现细粒度像素级推理。我们的方法有效性已在10个基准上得到验证，涵盖多种任务，包括像素级指代/分割以及图像/视频中以对象为中心的理解。此外，我们还设计了一个新颖的PixelQA任务，该任务同时要求指代、分割和问答，以验证我们方法的灵活性。
英文摘要：
Recent advances in Large Multi-modal Models (LMMs) have demonstrated their remarkable success as general-purpose multi-modal assistants, with particular focuses on holistic image- and video-language understanding. Conversely, less attention has been given to scaling fine-grained pixel-level understanding capabilities, where the models are expected to realize pixel-level alignment between visual signals and language semantics. Some previous studies have applied LMMs to related tasks such as region-level captioning and referring expression segmentation. However, these models are limited to performing either referring or segmentation tasks independently and fail to integrate these fine-grained perception capabilities into visual reasoning. To bridge this gap, we propose UniPixel, a large multi-modal model capable of flexibly comprehending visual prompt inputs and generating mask-grounded responses. Our model distinguishes itself by seamlessly integrating pixel-level perception with general visual understanding capabilities. Specifically, UniPixel processes visual prompts and generates relevant masks on demand, and performs subsequent reasoning conditioning on these intermediate pointers during inference, thereby enabling fine-grained pixel-level reasoning. The effectiveness of our approach has been verified on 10 benchmarks across a diverse set of tasks, including pixel-level referring/segmentation and object-centric understanding in images/videos. A novel PixelQA task that jointly requires referring, segmentation, and question answering is also designed to verify the flexibility of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
