面向基于预训练模型的持续学习中的高效且精确的遗忘服务
Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning
AI总结：
针对持续学习中顺序遗忘请求的挑战，提出基于解析方法的持续遗忘（ACU），通过最小二乘递归推导闭式解，实现高效精确的遗忘，保护历史数据隐私。
摘要：
在持续学习（CL）中，使用预训练模型（PTM）作为特征提取器已成为一种流行做法。结合解析分类器，基于PTM的方法在CL中实现了最先进的性能，追求非遗忘目标。同时，在大多数服务构建范式（例如移动群智感知（MCS））中，主动遗忘在CL阶段获得的特定知识也至关重要，其中移动边缘节点不断收集传感数据，不仅需要非遗忘适应，还需要特定知识遗忘以保护隐私。因此，当遗忘请求在CL中顺序出现时，产生了一个独特的问题，称为持续遗忘（CU）。然而，现有的遗忘方法专注于单次联合遗忘，在应用于CU时显得非常不足，包括（1）违反CL中的历史数据隐私，以及（2）容易被对抗性频繁请求淹没或降级。为了应对CU的挑战，我们提出了一种无梯度方法，称为解析持续遗忘（ACU），用于在基于PTM的CL中实现高效且精确的遗忘，同时保护历史数据隐私。针对每个遗忘请求，我们的ACU通过最小二乘法以可解释的方式递归推导解析（即闭式）解。通过精心设计，我们的ACU兼容样本级和类别级遗忘请求。理论和实验评估验证了我们的ACU在遗忘有效性、模型保真度和系统效率方面的优越性。
英文摘要：
In Continual Learning (CL), using a Pre-Trained Model (PTM) as the feature extractor has become a popular practice. Accompanied by analytic classifiers, the PTM-based methods have achieved state-of-the-art performance in CL, in pursuit of the non-forgetting goal. Meanwhile, actively forgetting specific knowledge acquired during the CL phase is also essential in most service construction paradigms, for example, Mobile Crowd Sensing (MCS), where mobile edge nodes continuously collect sensory data and demand not only non-forgetting adaptation but also specific knowledge forgetting for privacy preservation. Thus, a unique problem, called Continual Unlearning (CU), arises when the forgetting requests show sequentially in CL. However, existing unlearning methods focus on single-shot joint forgetting and prove highly inadequate when applied to CU, including (1) violating the historical data privacy in CL and (2) vulnerably being overwhelmed or degraded with adversarially frequent requests. To handle the challenges of CU, we propose a gradient-free approach, called Analytic Continual Unlearning (ACU), for efficient and exact forgetting with historical data privacy preservation in PTM-based CL. In response to each unlearning request, our ACU recursively derives the analytical (i.e., closed-form) solutions via least squares in an interpretable manner. By meticulous design, our ACU is compatible with both sample-level and class-level unlearning requests. The theoretical and experimental evaluations validate our ACU's superiority in unlearning effectiveness, model fidelity, and system efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
