通过解耦提示学习实现无重放类增量学习的连续蒸馏学习
Continual Distillation Learning for Rehearsal-Free Class-Incremental Learning via Decoupled Prompting
AI总结：
该研究针对无重放类增量学习，提出解耦连续蒸馏学习（D-CDL），通过解耦提示与蒸馏提升跨模型知识迁移能力，在多数据集实验中优于现有蒸馏基线。
摘要：
基于提示的持续学习通过调整可学习提示并冻结预训练的视觉Transformer（ViT）主干，在无重放类增量学习中展现出强大性能。然而，主干规模的影响尚未得到充分探索。我们观察到更大的ViT主干始终能带来更好的持续学习性能，这促使我们研究如何将这种能力从更大的模型迁移到更小的模型。在本文中，我们提出了连续蒸馏学习（CDL），这是一种用于无重放、基于提示的持续学习中的知识蒸馏新设置。我们发现传统蒸馏方法在CDL中仅能提供有限的增益，主要原因是特定任务的提示被迫同时编码持续适应和蒸馏知识，且缺乏用于跨任务知识迁移的持久机制。为解决该问题，我们提出了解耦连续蒸馏学习（D-CDL），其引入了持久的知识蒸馏提示（KD-prompts）和专用的KD分支，以明确将蒸馏与任务适应解耦。所提出的KD-prompts作为教师知识的全局载体跨任务传播，而原始提示则负责持续学习。D-CDL简单通用，可集成到各种基于提示的持续学习框架中。在Split CIFAR-100和Split ImageNet-R上针对四种代表性持续学习方法开展的大量实验表明，D-CDL始终优于现有蒸馏基线，并在不同的教师-学生设置下显著提升了学生模型的性能。
英文摘要：
Prompt-based continual learning has shown strong performance in rehearsal-free class-incremental learning by adapting learnable prompts while freezing a pre-trained Vision Transformer (ViT) backbone. However, the effect of backbone scale remains underexplored. We observe that larger ViT backbones consistently yield better continual learning performance, which motivates us to study how to transfer such capability from a larger model to a smaller one. In this paper, we introduce Continual Distillation Learning (CDL), a new setting for knowledge distillation in rehearsal-free prompt-based continual learning. We show that conventional distillation methods provide only limited gains in CDL, mainly because task-specific prompts are forced to encode both continual adaptation and distillation knowledge, while lacking a persistent mechanism for cross-task knowledge transfer. To address this problem, we propose Decoupled Continual Distillation Learning (D-CDL), which introduces persistent Knowledge-Distillation prompts (KD-prompts) and a dedicated KD branch to explicitly decouple distillation from task adaptation. The proposed KD-prompts are propagated across tasks as a global carrier of teacher knowledge, while the original prompts remain responsible for continual learning. D-CDL is simple, general, and can be integrated into various prompt-based continual learning frameworks. Extensive experiments on Split CIFAR-100 and Split ImageNet-R across four representative continual learning methods show that D-CDL consistently outperforms existing distillation baselines and substantially improves student performance under different teacher-student settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
