无止境的进化：统一多模态增量学习以实现持续全景感知
Evolving Without Ending: Unifying Multimodal Incremental Learning for Continual Panoptic Perception
AI总结：
本文提出了一种统一多模态和多任务持续学习的持续全景感知模型，通过跨模态编码器和可变知识继承模块解决灾难性遗忘问题，并在多任务增量场景中实现模型进化。
摘要：
持续学习（CL）是开发智能感知AI系统的一大挑战。然而，早期研究主要集中在单一任务的持续学习，这限制了多任务和多模态场景的潜力。除了众所周知的灾难性遗忘问题外，多任务持续学习还导致多模态对齐中的语义混淆，从而在增量训练步骤中导致严重的模型退化。在本文中，我们扩展CL到持续全景感知（CPP），整合多模态和多任务CL以通过像素级、实例级和图像级联合解释增强全面图像感知。我们正式化了多模态场景中的CL任务，并提出了一种端到端的持续全景感知模型。具体而言，CPP模型包含一个协作跨模态编码器（CCE）用于多模态嵌入。我们还提出了一种通过对比特征蒸馏和实例蒸馏的可变知识继承模块，以任务交互增强的方式解决灾难性遗忘问题。此外，我们提出了一种跨模态一致性约束，并开发了CPP+，确保在多任务增量场景下多模态语义对齐的模型更新。此外，我们提出的模型采用了一种非对称伪标签方法，使模型在没有示例回放的情况下实现进化。在多模态数据集和多种CL任务上的广泛实验表明，所提模型的优越性，特别是在细粒度CL任务中。
英文摘要：
Continual learning (CL) is a great endeavour in developing intelligent perception AI systems. However, the pioneer research has predominantly focus on single-task CL, which restricts the potential in multi-task and multimodal scenarios. Beyond the well-known issue of catastrophic forgetting, the multi-task CL also brings semantic obfuscation across multimodal alignment, leading to severe model degradation during incremental training steps. In this paper, we extend CL to continual panoptic perception (CPP), integrating multimodal and multi-task CL to enhance comprehensive image perception through pixel-level, instance-level, and image-level joint interpretation. We formalize the CL task in multimodal scenarios and propose an end-to-end continual panoptic perception model. Concretely, CPP model features a collaborative cross-modal encoder (CCE) for multimodal embedding. We also propose a malleable knowledge inheritance module via contrastive feature distillation and instance distillation, addressing catastrophic forgetting from task-interactive boosting manner. Furthermore, we propose a cross-modal consistency constraint and develop CPP+, ensuring multimodal semantic alignment for model updating under multi-task incremental scenarios. Additionally, our proposed model incorporates an asymmetric pseudo-labeling manner, enabling model evolving without exemplar replay. Extensive experiments on multimodal datasets and diverse CL tasks demonstrate the superiority of the proposed model, particularly in fine-grained CL tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
