Transfer learning is widely used for training deep neural networks (DNN) for building a powerful representation. Even after the pre-trained model is adapted for the target task, the representation performance of the feature extractor is retained to some extent. As the performance of the pre-trained model can be considered the private property of the owner, it is natural to seek the exclusive right of the generalized performance of the pre-trained weight. To address this issue, we suggest a new paradigm of transfer learning called disposable transfer learning (DTL), which disposes of only the source task without degrading the performance of the target task. To achieve knowledge disposal, we propose a novel loss named Gradient Collision loss (GC loss). GC loss selectively unlearns the source knowledge by leading the gradient vectors of mini-batches in different directions. Whether the model successfully unlearns the source task is measured by piggyback learning accuracy (PL accuracy). PL accuracy estimates the vulnerability of knowledge leakage by retraining the scrubbed model on a subset of source data or new downstream data. We demonstrate that GC loss is an effective approach to the DTL problem by showing that the model trained with GC loss retains the performance on the target task with a significantly reduced PL accuracy.
Data-Free Privacy-Preserving for LLMs via Model Inversion and Selective Unlearning
School of Software Technology, Zhejiang University(浙江大学软件技术学院);Zhejiang University(浙江大学);Hefei University of Technology(合肥工业大学);Institute of Fundamental and Transdisciplinary Research, Zhejiang University(浙江大学基础与交叉学科研究院);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江区)区块链与数据安全研究院);The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Zhejiang Key Laboratory of Big Data Intelligent Computing(浙江省大数据智能计算重点实验室)
Mechanism-Guided Selective Unlearning for RLVR-Induced Reasoning
School of Engineering, Institute of Science Tokyo, Japan(东京科学研究院工程学院);College of Control Science and Engineering, Zhejiang University, China(浙江大学控制科学与工程学院);Department of Electrical and Computer Engineering, National University of Singapore, Singapore(新加坡国立大学电子与计算机工程系)
MeGU: Machine-Guided Unlearning with Target Feature Disentanglement
School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学);Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);Department of Computer Science, Hong Kong Baptist University(计算机科学系,香港 Baptist 大学)
Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data
Department of Neurosurgery, Qilu Hospital of Shandong University and Institute of Brain and Brain-Inspired Science(齐鲁医院山东大学神经外科和脑科学与脑启发科学研究院);Department of Neurosurgery, The Affiliated Hospital of Qingdao University(青岛大学附属医院神经外科);Department of Neurosurgery, Peking Union Medical College Hospital(北京地坛医院神经外科);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);College of Artificial Intelligence, Dalian Maritime University(大连海事大学人工智能学院);University of Colorado Boulder(科罗拉多大学波尔德分校);Zhengzhou University(郑州大学);The University of Hong Kong(香港大学);Georgia Institute of Technology(佐治亚理工学院)
Not All Change Is Damage: Update-Matched Unlearning for Large Language Models
Technische Universität Berlin(柏林工业大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Massachusetts Boston(马萨诸塞大学波士顿分校)
Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
Department of Mathematics, University of California Davis, USA(加州大学戴维斯分校数学系);SLAC National Accelerator Laboratory, Stanford University, USA(斯坦福大学SLAC国家加速器实验室);Department of Computer Science, University of California Davis, USA(加州大学戴维斯分校计算机科学系)