SENSE：立体视图开放词汇语义分割
SENSE: Stereo OpEN Vocabulary SEmantic Segmentation
AI总结：
本文提出SENSE，通过立体视觉与视觉语言模型提升开放词汇语义分割的精度，实验显示在PhraseStereo数据集上平均精度提升2.9%，并在Cityscapes和KITTI上取得更好的结果。
摘要：
开放词汇语义分割使模型能够分割超出固定类别集的对象或图像区域，提供在动态环境中的灵活性。然而，现有方法往往依赖单视角图像，在遮挡和近物体边界的情况下难以实现空间精度。我们提出SENSE，即首个立体开放词汇语义分割工作，通过立体视觉与视觉语言模型提升开放词汇语义分割。通过引入立体图像对，我们引入几何线索以提高空间推理和分割精度。在PhraseStereo数据集上训练，我们的方法在短语 grounded 任务中表现强劲，并在零样本设置中展示泛化能力。在PhraseStereo上，我们展示了相对于基线方法的平均精度提升2.9%，以及相对于最佳竞争方法的提升0.76%。SENSE还在Cityscapes上相对于基线工作提供了+3.5%的mIoU相对提升，在KITTI上达到+18%的提升。通过联合推理语义和几何，SENSE支持从自然语言准确理解场景，这对自主机器人和智能交通系统至关重要。
英文摘要：
Open-vocabulary semantic segmentation enables models to segment objects or image regions beyond fixed class sets, offering flexibility in dynamic environments. However, existing methods often rely on single-view images and struggle with spatial precision, especially under occlusions and near object boundaries. We propose SENSE, the first work on Stereo OpEN Vocabulary SEmantic Segmentation, which leverages stereo vision and vision-language models to enhance open-vocabulary semantic segmentation. By incorporating stereo image pairs, we introduce geometric cues that improve spatial reasoning and segmentation accuracy. Trained on the PhraseStereo dataset, our approach achieves strong performance in phrase-grounded tasks and demonstrates generalization in zero-shot settings. On PhraseStereo, we show a +2.9% improvement in Average Precision over the baseline method and +0.76% over the best competing method. SENSE also provides a relative improvement of +3.5% mIoU on Cityscapes and +18% on KITTI compared to the baseline work. By jointly reasoning over semantics and geometry, SENSE supports accurate scene understanding from natural language, essential for autonomous robots and Intelligent Transportation Systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
