arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09593cs.CV
收藏

开放词汇分割基准中的名称翻新

Renovating Names in Open-Vocabulary Segmentation Benchmarks

Bosch IoC Lab(博世物联网计算实验室);University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心);Autonomous Vision Group, University of Tübingen(蒂宾根大学自主视觉组);ETH Zurich(苏黎世联邦理工学院);MPI for Intelligent Systems, Tübingen(蒂宾根智能系统马克斯·普朗克研究所);Bosch Center for Artificial Intelligence(博世人工智能中心)

Haiwen Huang, Songyou Peng, Dan Zhang, Andreas Geiger

AI总结:

针对开放词汇分割基准中类别名称不精确的问题,提出RENOVATE框架翻新名称,提升模型性能达15%并改善评估,代码与重标注将开源。

摘要:

名称对于人类认知和视觉-语言模型都至关重要。开放词汇模型利用类别名称作为文本提示,以泛化到训练中未见过的类别。然而,在现有数据集中,这些名称的精确性往往被忽视。在本文中,我们通过提出一个用于开放词汇分割基准中名称“翻新”的框架(RENOVATE)来解决这一未被充分探索的问题。我们的框架具有一个重命名模型,该模型能提升每个视觉片段的名称质量。通过实验,我们证明翻新后的名称有助于训练更强的开放词汇模型,相对改进高达15%,并通过提高数据质量显著提升训练效率。我们还表明,翻新后的名称通过更好地衡量误分类和实现细粒度模型分析来改进评估。我们将向研究社区提供我们的代码以及几个流行分割数据集(MS COCO、ADE20K、Cityscapes)的重新标注。

英文摘要:

Names are essential to both human cognition and vision-language models. Open-vocabulary models utilize class names as text prompts to generalize to categories unseen during training. However, the precision of these names is often overlooked in existing datasets. In this paper, we address this underexplored problem by presenting a framework for "renovating" names in open-vocabulary segmentation benchmarks (RENOVATE). Our framework features a renaming model that enhances the quality of names for each visual segment. Through experiments, we demonstrate that our renovated names help train stronger open-vocabulary models with up to 15% relative improvement and significantly enhance training efficiency with improved data quality. We also show that our renovated names improve evaluation by better measuring misclassification and enabling fine-grained model analysis. We will provide our code and relabelings for several popular segmentation datasets (MS COCO, ADE20K, Cityscapes) to the research community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑