原始特征生成与语义相关对齐用于通用零样本分割
Primitive Generation and Semantic-related Alignment for Universal Zero-Shot Segmentation
AI总结：
本文提出一种基于生成模型和语义相关对齐的通用零样本分割方法，通过原始特征组装与特征解耦，实现全景、实例和语义分割的最先进性能。
摘要：
本文研究通用零样本分割，旨在无需任何训练样本的情况下，对新型类别实现全景分割、实例分割和语义分割。这种零样本分割能力依赖于语义空间中的类间关系，将已见类别学到的视觉知识迁移到未见类别。因此，理想的做法是很好地桥接语义-视觉空间，并将语义关系应用于视觉特征学习。我们引入一个生成模型来合成未见类别的特征，该模型连接了语义空间和视觉空间，并解决了未见训练数据缺乏的问题。此外，为了缓解语义空间与视觉空间之间的域差距，首先，我们利用学习到的原始特征（每个原始特征包含与类别相关的细粒度属性）增强原始生成器，并通过选择性组装这些原始特征来合成未见特征。其次，我们提出将视觉特征解耦为语义相关部分和语义无关部分，后者包含有用的视觉分类线索但与语义表示相关性较低。随后，要求语义相关视觉特征的类间关系与语义空间中的关系对齐，从而将语义知识转移到视觉特征学习中。所提出的方法在零样本全景分割、实例分割和语义分割上取得了令人印象深刻的最先进性能。代码可在 https://henghuiding.github.io/PADing/ 获取。
英文摘要：
We study universal zero-shot segmentation in this work to achieve panoptic, instance, and semantic segmentation for novel categories without any training samples. Such zero-shot segmentation ability relies on inter-class relationships in semantic space to transfer the visual knowledge learned from seen categories to unseen ones. Thus, it is desired to well bridge semantic-visual spaces and apply the semantic relationships to visual feature learning. We introduce a generative model to synthesize features for unseen categories, which links semantic and visual spaces as well as addresses the issue of lack of unseen training data. Furthermore, to mitigate the domain gap between semantic and visual spaces, firstly, we enhance the vanilla generator with learned primitives, each of which contains fine-grained attributes related to categories, and synthesize unseen features by selectively assembling these primitives. Secondly, we propose to disentangle the visual feature into the semantic-related part and the semantic-unrelated part that contains useful visual classification clues but is less relevant to semantic representation. The inter-class relationships of semantic-related visual features are then required to be aligned with those in semantic space, thereby transferring semantic knowledge to visual feature learning. The proposed approach achieves impressively state-of-the-art performance on zero-shot panoptic segmentation, instance segmentation, and semantic segmentation. Code is available at https://henghuiding.github.io/PADing/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
