PinPoint: 通过信息性内部点进行提示
PinPoint: Prompting with Informative Interior Points
AI总结：
针对指代图像分割中VLM与SAM结合时因提示模糊导致的性能差距，提出无需训练的确定性点选择器PinPoint，通过融合视觉线索选择稳定、信息丰富的内部点，在无训练下达到监督和强化学习方法的性能。
摘要：
现代指代图像分割流程将用于定位的视觉语言模型（VLM）与用于掩码生成的可提示分割器（如Segment Anything Model，SAM）相结合。先前该方案的无训练实例始终落后于微调和强化学习（RL）调优的专家，且不清楚差距来自VLM的定位、SAM的能力还是提示。我们表明差距主要由提示模糊性主导：VLM提出的边界框（bbox）让SAM猜测框内哪些像素属于表达式所指的对象。内部点是自然的消歧器，但它们的落点很重要；先前的工作依赖于朴素采样的点，这些点落在边界、干扰物和背景杂波上，甚至可能比单独使用bbox更差。有监督和RL调优的方法通过训练VLM预测更好的点来缩小这一差距；我们表明这种训练是不必要的。在五个内部点的匹配预算下，用稳定、信息丰富的点选择替换朴素采样，在RefCOCO/+/g上累积交并比（cIoU）提高了12-18个点，且每个模型固定。我们将这一观察转化为PinPoint，一个确定性的、无需训练的点选择器，它融合四个视觉线索为共识图，选择紧凑、空间多样且远离边界的点，并使用冻结的VLM标记每个点。无需任何任务特定训练，PinPoint在相同堆栈上匹配了有监督和RL调优的专家，同时每次查询仅调用两次VLM。
英文摘要：
Modern referring image segmentation pipelines couple a vision-language model (VLM) for grounding with a promptable segmenter such as the Segment Anything Model (SAM) for mask generation. Prior training-free instances of this recipe consistently trail fine-tuned and reinforcement-learning (RL)-tuned specialists, and it has been unclear whether the gap comes from the VLM's grounding, SAM's capacity, or the prompt. We show that the gap is dominated by prompt ambiguity: a VLM-proposed bounding box (bbox) leaves SAM to guess which pixels inside the bbox belong to the object the expression denotes. Interior points are the natural disambiguator, but where they fall matters; prior work relies on naively sampled points that land on boundaries, distractors, and background clutter, and can even hurt performance compared to the bbox alone. Supervised and RL-tuned methods close this gap by training a VLM to predict better points; we show that this training is unnecessary. At a matched budget of five interior points, replacing naive sampling with stable, informative point selection improves cumulative Intersection-over-Union (cIoU) by 12-18 points across RefCOCO/+/g, with every model fixed. We turn this observation into PinPoint, a deterministic, training-free point selector that fuses four visual cues into a consensus map, selects compact, spatially diverse points away from boundaries, and uses the frozen VLM to label each point. Without any task-specific training, PinPoint matches supervised and RL-tuned specialists on the same stack while issuing only two VLM calls per query.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
