通过学习与遗忘实现知识交换
Knowledge Swapping via Learning and Unlearning
AI总结:
本文提出 Knowledge Swapping 任务,依据学习与遗忘在特征层级上的相反规律,采用 Learning Before Forgetting 策略实现指定知识遗忘、必要知识保留与新知识获取,并在多类视觉任务中验证其效果。
摘要:
我们提出 Knowledge Swapping(知识交换)这一新任务,旨在选择性调控预训练模型的知识,使其能够同时遗忘用户指定信息、保留必要知识并获取新知识。通过分析连锁特征层级,我们发现增量学习通常从低层表征推进到高层语义,而遗忘则倾向于沿相反方向发生——从高层语义开始并向下延伸至低层特征。基于此,我们提出以 Learning Before Forgetting(先学习后遗忘)策略对知识交换任务进行基准评测。在图像分类、目标检测和语义分割等多种任务上的综合实验验证了该策略的有效性。源代码可在 https://github.com/xingmingyu123456/KnowledgeSwapping 获取。
英文摘要:
We introduce \textbf{Knowledge Swapping}, a novel task designed to selectively regulate knowledge of a pretrained model by enabling the forgetting of user\-specified information, retaining essential knowledge, and acquiring new knowledge simultaneously. By delving into the analysis of knock-on feature hierarchy, we find that incremental learning typically progresses from low\-level representations to higher\-level semantics, whereas forgetting tends to occur in the opposite direction\-starting from high-level semantics and moving down to low-level features. Building upon this, we propose to benchmark the knowledge swapping task with the strategy of \textit{Learning Before Forgetting}. Comprehensive experiments on various tasks like image classification, object detection, and semantic segmentation validate the effectiveness of the proposed strategy. The source code is available at \href{https://github.com/xingmingyu123456/KnowledgeSwapping}{https://github.com/xingmingyu123456/KnowledgeSwapping}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。