量子门任务交互知识蒸馏用于基于预训练模型的类增量学习
Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning
AI总结：
本文提出量子门任务交互知识蒸馏框架，通过量子门引导任务间知识转移，缓解多任务子空间纠缠问题，提升类增量学习性能。
摘要：
类增量学习（CIL）旨在从连续任务流中积累知识并构建统一分类器。尽管预训练模型（PTMs）在CIL中表现优异，但仍面临多任务子空间纠缠问题，导致任务路由参数校准不佳或任务级表示固定时出现灾难性遗忘。为此，我们提出一种新颖的量子门任务交互知识蒸馏（QKD）框架，利用量子门引导任务间知识转移。具体而言，我们引入量子门任务调节门机制，建模任务嵌入之间的关系依赖，动态捕捉样本到任务的相关性，用于流式任务的联合训练和推理。受量子门输出引导，我们执行任务交互知识蒸馏，利用任务嵌入级相关权重从旧到新适配器，使模型在独立任务子空间间建立表示桥梁。大量实验表明，QKD有效缓解遗忘并实现最先进的性能。
英文摘要：
Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks and construct a unified classifier over all seen classes. Although pretrained models (PTMs) have shown promising performance in CIL, they still struggle with the entanglement of multi-task subspaces, leading to catastrophic forgetting when task routing parameters are poorly calibrated or task-level representations are rigidly fixed. To address this issue, we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer. Specifically, we introduce a quantum-gated task modulation gating mechanism to model the relational dependencies among task embedding, dynamically capturing the sample-to-task relevance for both joint training and inference across streaming tasks. Guided by the quantum gating outputs, we perform task-interaction knowledge distillation guided by these task-embedding-level correlation weights from old to new adapters, enabling the model to bridge the representation gaps between independent task subspaces. Extensive experiments demonstrate that QKD effectively mitigates forgetting and achieves state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
