回答前分割：用于多模态大语言模型视觉推理的像素定位
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning
AI总结：
本文针对多模态大语言模型视觉推理，提出SegAnswer方法，将放大单元从边界框转为像素级分割掩码，能精准定位感兴趣区域，过滤冗余信息，与视觉令牌构建方式更契合，经多基准测试验证了其在像素定位及分割任务上的性能。
摘要：
多模态大语言模型（MLLMs）的最新进展已从静态感知发展到交错式视觉语言推理，即所谓的“用图像思考”。这种推理过程的一个基本操作是放大感兴趣区域（通常用边界框表示）以获取更精细的视觉细节。本文提出了回答前分割（SegAnswer）方法，将放大单元从流行的边界框转变为像素级分割掩码。通过使用细粒度掩码将目标区域与杂乱环境隔离开，分割后的视觉输入产生更精确的感兴趣区域，有效过滤掉冗余背景和干扰对象。此外，分割后的视觉输入的离散块与MLLMs通过位置嵌入构建视觉令牌的方式更无缝地对齐。在实验中，我们在包括高分辨率感知、一般感知和幻觉等各种基准上评估了SegAnswer。它实现了持续改进，并且在分割任务上也表现出相当的性能，验证了其可靠像素定位的能力。
英文摘要：
Recent advancements in Multimodal Large Language Models (MLLMs) have evolved from static perception to interleaved visual-language reasoning, often referred to as ``thinking with images''. A basic operation in this reasoning process is to zoom in on regions of interest (often represented with bounding boxes) to acquire finer visual details. In this paper, we propose \textbf{Seg}mentation before \textbf{Answer}ing (SegAnswer), which shifts the unit of zoom-in from the popular bounding box to pixel-level segmentation mask. By employing fine-grained masks to isolate the target area from cluttered environments, segmented visual input yields a more precise region of interest, effectively filtering out redundant background and interfering objects. Furthermore, the discrete patches of segmented visual input align more seamlessly with how MLLMs structure visual tokens via positional embeddings. In experiments, we evaluate SegAnswer across diverse benchmarks, including high-resolution perception, general perception, and hallucination. It achieves consistent improvements and also exhibits considerable performance on segmentation tasks, validating its capability for reliable pixel grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
