NICE：通过级联协作学习改进全景叙述检测与分割
NICE: Improving Panoptic Narrative Detection and Segmentation with Cascading Collaborative Learning
AI总结：
提出统一框架NICE，通过基于掩码重心的级联模块CGA和BDL联合学习全景叙述检测与分割，解决预测冲突并实现任务互补，大幅超越现有方法。
摘要：
全景叙述检测（PND）和分割（PNS）是两项具有挑战性的任务，涉及根据长篇叙述描述识别并定位图像中的多个目标。本文提出了一种统一且有效的框架NICE，可联合学习这两项全景叙述识别任务。现有视觉定位任务采用双分支范式，但直接将其应用于PND和PNS会因其固有的多对多对齐特性导致预测冲突。为解决此问题，我们引入了两个基于掩码重心的级联模块，即坐标引导聚合（CGA）和重心驱动定位（BDL），分别负责分割和检测。通过以分割重心为锚点将PNS和PND串联，该方法自然地对齐了两项任务，使其互补以提升性能。具体而言，CGA提供重心作为检测参考，减少了BDL对大量候选框的依赖。BDL利用其优异特性区分不同实例，提升了CGA的分割性能。大量实验表明，NICE大幅超越现有方法，在PND和PNS上分别比最优方法提升了4.1%和2.9%。这些结果验证了所提协作学习策略的有效性。项目已公开发布于 https://github.com/Mr-Neko/NICE。
英文摘要：
Panoptic Narrative Detection (PND) and Segmentation (PNS) are two challenging tasks that involve identifying and locating multiple targets in an image according to a long narrative description. In this paper, we propose a unified and effective framework called NICE that can jointly learn these two panoptic narrative recognition tasks. Existing visual grounding tasks use a two-branch paradigm, but applying this directly to PND and PNS can result in prediction conflict due to their intrinsic many-to-many alignment property. To address this, we introduce two cascading modules based on the barycenter of the mask, which are Coordinate Guided Aggregation (CGA) and Barycenter Driven Localization (BDL), responsible for segmentation and detection, respectively. By linking PNS and PND in series with the barycenter of segmentation as the anchor, our approach naturally aligns the two tasks and allows them to complement each other for improved performance. Specifically, CGA provides the barycenter as a reference for detection, reducing BDL's reliance on a large number of candidate boxes. BDL leverages its excellent properties to distinguish different instances, which improves the performance of CGA for segmentation. Extensive experiments demonstrate that NICE surpasses all existing methods by a large margin, achieving 4.1% for PND and 2.9% for PNS over the state-of-the-art. These results validate the effectiveness of our proposed collaborative learning strategy. The project of this work is made publicly available at https://github.com/Mr-Neko/NICE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
