Generative AI technologies have been deployed in many places, such as (multimodal) large language models and vision generative models. Their remarkable performance should be attributed to massive training data and emergent reasoning abilities. However, the models would memorize and generate sensitive, biased, or dangerous information originated from the training data especially those from web crawl. New machine unlearning (MU) techniques are being developed to reduce or eliminate undesirable knowledge and its effects from the models, because those that were designed for traditional classification tasks could not be applied for Generative AI. We offer a comprehensive survey on many things about MU in Generative AI, such as a new problem formulation, evaluation methods, and a structured discussion on the advantages and limitations of different kinds of MU techniques. It also presents several critical challenges and promising directions in MU research. A curated list of readings can be found: https://github.com/franciscoliu/GenAI-MU-Reading.
A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(莫扎德大学人工智能学院);Fraunhofer Institute for Open Communication Systems (FOKUS)(弗劳恩霍夫开放通信系统研究所);Technical University of Munich(慕尼黑技术大学);Technical University of Darmstadt(达姆施塔特技术大学);University of Toronto(多伦多大学)
TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation
Stony Brook University(石英布鲁克大学);University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);Columbia University(哥伦比亚大学);Institute of Science and Technology Austria(奥地利科学技术研究院);Sapienza University of Rome(罗马大学 sapienza);National University of Singapore(新加坡国立大学)
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research
The GenLaw Center(GenLaw中心);Microsoft Research(微软研究院);Stanford University(斯坦福大学);Google DeepMind(谷歌DeepMind);Center for Democracy & Technology(民主与科技中心);Princeton(普林斯顿);Google(谷歌);University of Washington(华盛顿大学);University of Michigan(密歇根大学);Cornell Tech(康奈尔科技);Cornell Law School(康奈尔法学院);Cornell University(康奈尔大学);Lighthouse;Stanford Law School(斯坦福法学院);UC Berkeley(伯克利大学);Independent(独立研究者);Northeastern University(东北大学);Harvard Business School(哈佛商学院);W. Virginia University College of Law(维珍尼亚大学法学院)
Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy
University of Southern Queensland(南昆士兰大学);Lingnan University(岭南大学);Wuhan University of Technology(武汉理工大学);University of Electronic Science and Technology of China(电子科技大学);Hong Kong Polytechnic University(香港理工大学)
本文系统综述机器遗忘的最新方法、指标、数据集与挑战,并指出其在隐私、安全、公平及可信 AI 中的未来方向。
Griffith University(格里菲斯大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);University of Bremen(不来梅大学);Hanoi University of Science and Technology(河内科技大学);The University of Queensland(昆士兰大学)
School of Advanced Interdisciplinary Sciences, UCAS(UCAS交叉学科研究院);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院);Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation
Guilin University of Electronic Technology(桂林电子科技大学);University of Southampton(南安普顿大学);Jinan University(暨南大学);Nanjing University of Science and Technology(南京理工大学)
LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Technical University of Munich(慕尼黑技术大学);Siemens AG(西门子股份公司);University of Oxford(牛津大学)
HKUST(香港科技大学);University of Cambridge(剑桥大学);Central South University(中南大学);Shanghai Jiaotong University(上海交通大学);SenseTime Research(商汤科技研究院);Tsinghua University(清华大学)
Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs
Dept. of Computer Science & Munich Data Science Institute, Technical University of Munich(计算机科学系及慕尼黑数据科学研究所,技术大学慕尼黑);Mila, Université de Montréal(蒙特利尔大学Mila)
Rethinking Machine Unlearning for Large Language Models
Michigan State University(密歇根州立大学);IBM Research(IBM研究院);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance Research(字节跳动研究院);Stanford University(斯坦福大学)
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Nanjing University of Science and Technology(南京理工大学);Xidian University(西安电子科技大学);CSIRO(联邦科学与工业研究组织);University of Western Australia(西澳大学);University of Technology Sydney(悉尼科技大学)
Operationalising the Right to be Forgotten in LLMs: A Lightweight Sequential Unlearning Framework for Privacy-Aligned Deployment in Politically Sensitive Environments
Department of Mathematics(数学系);Munster Technological University(穆恩斯特技术大学);Department of Computer Science(计算机科学系)
State Key Laboratory of Integrated Services Networks (ISN)(综合业务网理论及关键技术国家重点实验室);Shaanxi Key Laboratory of Network and System Security, Xidian University(西安电子科技大学陕西省网络与系统安全重点实验室);School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);Ant Group(蚂蚁集团);College of Computer and Data Science, Fuzhou University(福州大学计算机与数据科学学院)