缓解对象性偏差和区域到文本对齐问题以实现开放词汇全景分割
Mitigating Objectness Bias and Region-to-Text Misalignment for Open-Vocabulary Panoptic Segmentation
AI总结：
本文提出OVRCOAT框架，通过CLIP条件对象性调整和开放词汇掩码到文本细化，解决开放词汇全景分割中的对象性偏差和区域对齐问题，提升分割性能。
摘要：
开放词汇全景分割仍受两个耦合问题阻碍：(i) 掩码选择偏差，即在封闭词汇上训练的对象性头会抑制训练中未观察到的类别的掩码；(ii) 视觉-语言模型如CLIP在局部分割上的有限理解，这些模型优化于全局图像分类而非局部分割。我们引入OVRCOAT，一个简单且模块化的框架，同时解决这两个问题。首先，CLIP条件对象性调整(COAT)更新背景/前景概率，保留高质量的非词汇对象掩码。其次，开放词汇掩码到文本细化(OVR)加强CLIP的区域级对齐，以提高对已见和未见类别的分类，其内存成本显著低于先前的微调方案。两个组件共同改进对象性估计和掩码识别，产生一致的全景增益。尽管其简单性，OVRCOAT在ADE20K上达到新的状态-of-the-art (+5.5% PQ)，并在Mapillary Vistas和Cityscapes上分别获得+7.1%和+3%的PQ增益。代码可在：https://github.com/nickormushev/OVRCOAT获取。
英文摘要：
Open-vocabulary panoptic segmentation remains hindered by two coupled issues: (i) mask selection bias, where objectness heads trained on closed vocabularies suppress masks of categories not observed in training, and (ii) limited regional understanding in vision-language models such as CLIP, which were optimized for global image classification rather than localized segmentation. We introduce OVRCOAT, a simple, modular framework that tackles both. First, a CLIP-conditioned objectness adjustment (COAT) updates background/foreground probabilities, preserving high-quality masks for out-of-vocabulary objects. Second, an open-vocabulary mask-to-text refinement (OVR) strengthens CLIP's region-level alignment to improve classification of both seen and unseen classes with markedly lower memory cost than prior fine-tuning schemes. The two components combine to jointly improve objectness estimation and mask recognition, yielding consistent panoptic gains. Despite its simplicity, OVRCOAT sets a new state of the art on ADE20K (+5.5% PQ) and delivers clear gains on Mapillary Vistas and Cityscapes (+7.1% and +3% PQ, respectively). The code is available at: https://github.com/nickormushev/OVRCOAT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
