持续全景感知：面向遥感图像多模态增量解译
Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images
AI总结：
本文提出持续全景感知模型CPP，通过多任务联合学习、跨模态编码与任务交互式知识蒸馏，在无样本记忆条件下缓解遥感图像持续解译中的灾难性遗忘，并实现端到端多模态全景感知。
摘要：
持续学习（CL）打破了单向训练方式，使模型能够不断适应新数据、新语义和新任务。然而，当前的持续学习方法主要关注单一任务。此外，由于缺乏旧数据，持续学习模型受到灾难性遗忘和语义漂移的困扰，而遥感解译中精细粒度语义复杂，这一问题尤为常见。本文提出持续全景感知（CPP），这是一个统一的持续学习模型，利用多任务联合学习，涵盖像素级分类、实例级分割和图像级感知，以实现遥感图像的通用解译。具体而言，我们提出一个协作式跨模态编码器（CCE）来提取输入图像特征，同步支持像素分类和文本描述生成。为了在没有样本记忆的情况下继承旧模型的知识，我们提出一种任务交互式知识蒸馏（TKD）方法，利用跨模态优化和任务非对称伪标签（TPL）来缓解灾难性遗忘。此外，我们还提出一种联合优化机制，以实现端到端的多模态全景感知。在精细粒度全景感知数据集上的实验结果验证了所提模型的有效性，同时证明联合优化能够提升子任务持续学习效率，在全景质量上相对提升超过13%。
英文摘要：
Continual learning (CL) breaks off the one-way training manner and enables a model to adapt to new data, semantics and tasks continuously. However, current CL methods mainly focus on single tasks. Besides, CL models are plagued by catastrophic forgetting and semantic drift since the lack of old data, which often occurs in remote-sensing interpretation due to the intricate fine-grained semantics. In this paper, we propose Continual Panoptic Perception (CPP), a unified continual learning model that leverages multi-task joint learning covering pixel-level classification, instance-level segmentation and image-level perception for universal interpretation in remote sensing images. Concretely, we propose a collaborative cross-modal encoder (CCE) to extract the input image features, which supports pixel classification and caption generation synchronously. To inherit the knowledge from the old model without exemplar memory, we propose a task-interactive knowledge distillation (TKD) method, which leverages cross-modal optimization and task-asymmetric pseudo-labeling (TPL) to alleviate catastrophic forgetting. Furthermore, we also propose a joint optimization mechanism to achieve end-to-end multi-modal panoptic perception. Experimental results on the fine-grained panoptic perception dataset validate the effectiveness of the proposed model, and also prove that joint optimization can boost sub-task CL efficiency with over 13\% relative improvement on panoptic quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
