ECLIPSE：基于视觉提示微调的全景分割高效持续学习
ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning
AI总结：
ECLIPSE通过冻结基础模型参数并仅微调少量视觉提示嵌入，结合logit操作缓解错误传播与语义漂移，在ADE20K持续全景分割基准上实现了抗灾难性遗忘与合理可塑性的新最优性能。
摘要：
全景分割结合了语义分割和实例分割，是一项前沿的计算机视觉任务。尽管深度学习模型近期取得了进展，但现实应用的动态特性要求持续学习，即模型随时间适应新类别（可塑性）而不遗忘旧类别（灾难性遗忘）。当前的持续分割方法通常依赖知识蒸馏和伪标签等蒸馏策略，这些方法虽然有效，但会导致训练复杂度和计算开销增加。本文提出了一种新颖且高效的基于视觉提示微调的持续全景分割方法，称为ECLIPSE。我们的方法冻结基础模型参数，仅微调少量提示嵌入，同时解决灾难性遗忘和可塑性问题，并显著减少可训练参数。为缓解持续分割中固有的错误传播和语义漂移等挑战，我们提出logit操作以有效利用类别间的通用知识。在ADE20K持续全景分割基准上的实验证明了ECLIPSE的优越性，尤其是其对灾难性遗忘的鲁棒性和合理的可塑性，达到了新的最先进水平。代码可在https://github.com/clovaai/ECLIPSE获取。
英文摘要：
Panoptic segmentation, combining semantic and instance segmentation, stands as a cutting-edge computer vision task. Despite recent progress with deep learning models, the dynamic nature of real-world applications necessitates continual learning, where models adapt to new classes (plasticity) over time without forgetting old ones (catastrophic forgetting). Current continual segmentation methods often rely on distillation strategies like knowledge distillation and pseudo-labeling, which are effective but result in increased training complexity and computational overhead. In this paper, we introduce a novel and efficient method for continual panoptic segmentation based on Visual Prompt Tuning, dubbed ECLIPSE. Our approach involves freezing the base model parameters and fine-tuning only a small set of prompt embeddings, addressing both catastrophic forgetting and plasticity and significantly reducing the trainable parameters. To mitigate inherent challenges such as error propagation and semantic drift in continual segmentation, we propose logit manipulation to effectively leverage common knowledge across the classes. Experiments on ADE20K continual panoptic segmentation benchmark demonstrate the superiority of ECLIPSE, notably its robustness against catastrophic forgetting and its reasonable plasticity, achieving a new state-of-the-art. The code is available at https://github.com/clovaai/ECLIPSE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
