大语言模型（LLMs）已被发现会记忆训练文本序列，并在文本生成期间逐字复述这些序列。这一事实被认为是隐私及相关（如版权）问题的根源。因此，LLMs 中的遗忘（Unlearning）表现为设计新算法，以妥善处理这些已记忆数据的副作用，同时不损害模型的实用性。我们为实现这一目标提供了一个新视角，即每个待遗忘的文本序列在遗忘时应根据其在 LLM 内的记忆程度被区别对待。我们贡献了一种衡量遗忘质量的新指标，一种对抗攻击，表明缺乏这一视角的 SOTA 算法在隐私方面失效，以及两种分别基于梯度上升（Gradient Ascent）和任务算术（Task Arithmetic）的新遗忘方法。随后，在一系列广泛的 NLP 任务上进行的综合性能评估绘制了解空间，识别了在不同模型容量规模和遗忘集大小下的最佳方案，并量化了新方法的收益。
英文摘要：
LLMs have been found to memorize training textual sequences and regurgitate verbatim said sequences during text generation time. This fact is known to be the cause of privacy and related (e.g., copyright) problems. Unlearning in LLMs then takes the form of devising new algorithms that will properly deal with these side-effects of memorized data, while not hurting the model's utility. We offer a fresh perspective towards this goal, namely, that each textual sequence to be forgotten should be treated differently when being unlearned based on its degree of memorization within the LLM. We contribute a new metric for measuring unlearning quality, an adversarial attack showing that SOTA algorithms lacking this perspective fail for privacy, and two new unlearning methods based on Gradient Ascent and Task Arithmetic, respectively. A comprehensive performance evaluation across an extensive suite of NLP tasks then mapped the solution space, identifying the best solutions under different scales in model capacities and forget set sizes and quantified the gains of the new approaches.
Attention Smoothing Is All You Need For Unlearning
Department of Computer Science, Wayne State University（韦恩州立大学计算机科学系）；Department of Computer Science and Engineering, Michigan State University（密歇根州立大学计算机科学与工程系）；Institute for AI and Data Science, Wayne State University（韦恩州立大学人工智能与数据科学研究所）
MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；The Hong Kong Polytechnic University（香港理工大学）；Huawei（华为）
Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机与人工智能学院）；BEDI Cloud（BEDI云公司）；School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Transnational Law, Peking University（北京大学国际法学院）
School of Advanced Interdisciplinary Sciences, UCAS（UCAS交叉学科研究院）；Academy of Mathematics and Systems Science, CAS（中国科学院数学与系统科学研究院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tongji University（同济大学）
Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting
Faculty of Information Technology, Monash University（墨尔本大学信息科技学院）；Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)（教育部计算能力网络与信息安全重点实验室，山东计算机科学中心，齐鲁大学（山东科学院））；Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science（山东省计算能力互联网与服务计算重点实验室，山东省计算机科学基础研究中心）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）
DP2Unlearning: An Efficient and Guaranteed Unlearning Framework for LLMs
Universitat Rovira i Virgili, Department of Computer Engineering（罗维拉-维格利大学计算机工程系）；Mathematics, CYBERCAT-Center for Cybersecurity Research of Catalonia（数学，加泰罗尼亚网络安全研究中心）
The Frontier of Data Erasure: Machine Unlearning for Large Language Models
Data61, Commonwealth Scientific and Industrial Research Organization（联邦科学与工业研究组织Data61）；School of Systems & Computing, University of New South Wales（新南威尔士大学系统与计算学院）；School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；College of Computing & Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Textual Unlearning Gives a False Sense of Unlearning
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, P. R. China（区块链与数据安全国家重点实验室，浙江大学，中华人民共和国）；School of Cyber Science and Technology, Zhejiang University, P. R. China（网络安全与技术学院，浙江大学，中华人民共和国）；ETH Zurich, Switzerland（苏黎世联邦理工学院，瑞士）
Can Large Language Models Reinvent Foundational Algorithms?
Xiongan AI Institute（雄安人工智能研究院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai Qi Zhi Institute（上海期智研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）
University of Chicago（芝加哥大学）；Argonne National Laboratory（阿贡国家实验室）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Dartmouth College（达特茅斯学院）；International Computer Science Institute（国际计算机科学研究所）；University of California, Berkeley（加州大学伯克利分校）
