PanopticPartFormer++：面向全景部件分割的统一解耦视图
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
AI总结：
本文提出端到端统一框架PanopticPartFormer++，通过解耦部件与全景特征、部件-整体交叉注意力及新指标PWQ，解决全景部件分割中任务割裂与度量偏差问题，并在两个数据集上取得最优结果。
摘要：
全景部件分割（PPS）将全景分割和部件分割统一为一个任务。以往的工作使用相互独立的方法处理物体、背景和部件预测，缺乏共享计算和任务关联。我们旨在从架构层面统一这些任务，设计出第一个端到端统一框架 Panoptic-PartFormer。此外，我们发现以往的度量指标 PartPQ 偏向于 PQ。为解决这两个问题，我们首先设计了一种元架构，分别解耦部件特征与物体/背景特征。我们将物体、背景和部件建模为对象查询，并直接学习将三种形式的预测统一优化为一个掩码预测和分类问题。我们将该模型命名为 Panoptic-PartFormer。其次，我们提出了一种新的度量指标 Part-Whole Quality（PWQ），能够更好地从像素区域和部件-整体角度衡量该任务，同时解耦部件分割和全景分割的误差。第三，受 Mask2Former 启发，基于我们的元架构，我们提出了 Panoptic-PartFormer++，并设计了一种新的部件-整体交叉注意力方案，以进一步提升部件分割质量。我们设计了一种使用掩码交叉注意力的部件-整体交互方法。最后，大量的消融研究和分析证明了 Panoptic-PartFormer 和 Panoptic-PartFormer++ 的有效性。与之前的 Panoptic-PartFormer 相比，我们的 Panoptic-PartFormer++ 在 Cityscapes PPS 数据集上实现了 2% 的 PartPQ 和 3% 的 PWQ 提升，在 Pascal Context PPS 数据集上实现了 5% 的 PartPQ 提升。在两个数据集上，Panoptic-PartFormer++ 均取得了新的最先进结果。我们的模型可以作为强有力的基线，助力未来 PPS 研究。源代码和训练好的模型将在 https://github.com/lxtGH/Panoptic-PartFormer 提供。
英文摘要：
Panoptic Part Segmentation (PPS) unifies panoptic and part segmentation into one task. Previous works utilize separate approaches to handle things, stuff, and part predictions without shared computation and task association. We aim to unify these tasks at the architectural level, designing the first end-to-end unified framework, Panoptic-PartFormer. Moreover, we find the previous metric PartPQ biases to PQ. To handle both issues, we first design a meta-architecture that decouples part features and things/stuff features, respectively. We model things, stuff, and parts as object queries and directly learn to optimize all three forms of prediction as a unified mask prediction and classification problem. We term our model as Panoptic-PartFormer. Second, we propose a new metric Part-Whole Quality (PWQ), better to measure this task from pixel-region and part-whole perspectives. It also decouples the errors for part segmentation and panoptic segmentation. Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further. We design a new part-whole interaction method using masked cross attention. Finally, extensive ablation studies and analysis demonstrate the effectiveness of both Panoptic-PartFormer and Panoptic-PartFormer++. Compared with previous Panoptic-PartFormer, our Panoptic-PartFormer++ achieves 2% PartPQ and 3% PWQ improvements on the Cityscapes PPS dataset and 5% PartPQ on the Pascal Context PPS dataset. On both datasets, Panoptic-PartFormer++ achieves new state-of-the-art results. Our models can serve as a strong baseline and aid future research in PPS. The source code and trained models will be available at~\url{https://github.com/lxtGH/Panoptic-PartFormer}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
