KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment
The Chinese University of Hong Kong(香港中文大学);MoE Key Laboratory of High Confidence Software Technologies(教育部高可信软件技术重点实验室);The University of Queensland(昆士兰大学)
Recent legislation of the "right to be forgotten" has led to the interest in machine unlearning, where the learned models are endowed with the function to forget information about specific training instances as if they have never existed in the training set. Previous work mainly focuses on computer vision scenarios and largely ignores the essentials of unlearning in NLP field, where text data contains more explicit and sensitive personal information than images. In this paper, we propose a general unlearning framework called KGA to induce forgetfulness. Different from previous work that tries to recover gradients or forces models to perform close to one specific distribution, KGA maintains distribution differences (i.e., knowledge gap). This relaxes the distribution assumption. Furthermore, we first apply the unlearning method to various NLP tasks (i.e., classification, translation, response generation) and propose several unlearning evaluation metrics with pertinence. Experiments on large-scale datasets show that KGA yields comprehensive improvements over baselines, where extensive analyses further validate the effectiveness of KGA and provide insight into unlearning for NLP tasks.
Beyond Entity Alignment: Towards Complete Knowledge Graph Alignment via Entity-Relation Synergy
School of Information Science and Engineering, Yanshan University(燕山大学信息科学与工程学院);Beijing University of Posts and Telecommunications(北京邮电大学);Beihang University(北京航空航天大学)
Safety Alignment via Constrained Knowledge Unlearning
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));University of Macau, China(澳门大学);Nankai University, China(南开大学);Zhejiang University, China(浙江大学);Peng Cheng Laboratory, China(鹏城实验室);Shaqra University, Saudi Arabia(沙克拉大学)
Leveraging Machine Unlearning for Cost-Efficient Preference Alignment
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Communication Engineering, Hangzhou Dianzi University(杭州电子科技大学通信工程学院)
Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU
Aalborg University(奥尔堡大学);University of Barcelona(巴塞罗那大学);Computer Vision Center(计算机视觉中心);Milestone Systems(里程碑系统公司);Pioneer Center for Artificial Intelligence(先锋人工智能中心)
Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
The Hong Kong Polytechnic University(香港理工大学);Carnegie Mellon University(卡内基梅隆大学);University of California, Santa Cruz(加州大学圣克ruz分校);Huawei Technologies(华为技术有限公司);Research Centre for Privacy and Security Technologies in Future Smart Systems, PolyU(未来智能系统中的隐私与安全技术研究中心,PolyU)
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
TUM University Hospital(慕尼黑工业大学医院);Technical University of Munich (TUM)(慕尼黑技术大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Department of Computing, Imperial College London(伦敦帝国学院计算系)