从易到难：学习课程式形状感知特征以实现鲁棒的全景场景图生成
From Easy to Hard: Learning Curricular Shape-aware Features for Robust Panoptic Scene Graph Generation
AI总结：
针对现有全景场景图生成方法忽略形状感知特征的问题，提出模型无关的课程式形状感知特征学习策略，按认知难度分阶段引入特征并结合知识蒸馏，在鲁棒和零样本PSG任务上性能显著优于现有方法。
摘要：
全景场景图生成（Panoptic Scene Graph Generation, PSG）旨在基于全景分割掩码生成全面的图结构表示。尽管PSG领域已取得显著进展，但现有几乎所有方法都忽略了形状感知特征的重要性，这类特征本质上聚焦于物体的轮廓与边界。为弥补这一空白，我们提出一种适用于PSG的模型无关型课程式形状感知特征（Curricular shApe-aware FEature, CAFE）学习策略。具体而言，我们将形状感知特征（即掩码特征和边界特征）引入PSG，突破了仅依赖边界框（bbox）特征的局限。此外，受人类认知机制启发，我们提出以从易到难的方式整合形状感知特征。为此，我们根据认知学习难度将谓词分为三组，并相应地将训练过程划分为三个阶段。每个阶段使用专门的关系分类器来区分特定组别的谓词。随着谓词学习难度提升，这些分类器将配备复杂度逐步升高的特征。我们还引入知识蒸馏机制以保留早期阶段习得的知识。由于具备模型无关的特性，CAFE可无缝集成到任意PSG模型中。在鲁棒PSG和零样本PSG两种设置下，针对两项PSG任务开展的大量实验与消融研究均验证了我们提出的CAFE的优越性与鲁棒性，其性能大幅超越现有最先进方法。
英文摘要：
Panoptic Scene Graph Generation (PSG) aims to generate a comprehensive graph-structure representation based on panoptic segmentation masks. Despite remarkable progress in PSG, almost all existing methods neglect the importance of shape-aware features, which inherently focus on the contours and boundaries of objects. To bridge this gap, we propose a model-agnostic Curricular shApe-aware FEature (CAFE) learning strategy for PSG. Specifically, we incorporate shape-aware features (i.e., mask features and boundary features) into PSG, moving beyond reliance solely on bbox features. Furthermore, drawing inspiration from human cognition, we propose to integrate shape-aware features in an easy-to-hard manner. To achieve this, we categorize the predicates into three groups based on cognition learning difficulty and correspondingly divide the training process into three stages. Each stage utilizes a specialized relation classifier to distinguish specific groups of predicates. As the learning difficulty of predicates increases, these classifiers are equipped with features of ascending complexity. We also incorporate knowledge distillation to retain knowledge acquired in earlier stages. Due to its model-agnostic nature, CAFE can be seamlessly incorporated into any PSG model. Extensive experiments and ablations on two PSG tasks under both robust and zero-shot PSG have attested to the superiority and robustness of our proposed CAFE, which outperforms existing state-of-the-art methods by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
