Pre-trained Large Language Models (LLMs) have demonstrated remarkable capabilities but also pose risks by learning and generating copyrighted material, leading to significant legal and ethical concerns. In real-world scenarios, model owners need to continuously address copyright infringement as new requests for content removal emerge at different time points. This leads to the need for sequential unlearning, where copyrighted content is removed sequentially as new requests arise. Despite its practical relevance, sequential unlearning in the context of copyright infringement has not been rigorously explored in existing literature. To address this gap, we propose Stable Sequential Unlearning (SSU), a novel framework designed to unlearn copyrighted content from LLMs over multiple time steps. Our approach works by identifying and removing specific weight updates in the model's parameters that correspond to copyrighted content. We improve unlearning efficacy by introducing random labeling loss and ensuring the model retains its general-purpose knowledge by adjusting targeted parameters. Experimental results show that SSU achieves an effective trade-off between unlearning efficacy and general-purpose language abilities, outperforming existing baselines.
Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks
Pine AI;The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Duke University(杜克大学);University of Michigan(密歇根大学);Stevens Institute of Technology(史蒂文斯理工学院);Johns Hopkins University(约翰霍普金斯大学);University of Notre Dame(圣母大学);The Pennsylvania State University(宾夕法尼亚州立大学)
Buchmann Faculty of Law, Tel Aviv University(特拉维夫大学法学院);Blavatnik School of Computer Science(比拉维克计算机科学学院);AI , Tel Aviv University(人工智能,特拉维夫大学);School of Electrical and Computer Engineering(电气与计算机工程学院);Departments of Mathematics, Computer Science(数学与计算机科学系);Decision Sciences Technion– Israel Institute of Technology(决策科学技术学院——以色列理工学院)
Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Matter
Shanghai University of Finance and Economics(上海金融学院);Alibaba Group(阿里巴巴集团);Southern University of Science and Technology(南方科技大学);Beihang University(北航)
Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture
School of Computer Science and the State Key Laboratory of Multimedia Information Processing, Peking University(计算机科学系和多媒体信息处理国家重点实验室,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);department of Electrical and Computer Engineering, University of California, Santa Cruz(电气与计算机工程系,加州大学圣克鲁兹分校);Alibaba DAMO Academy, Beijing(阿里巴巴达摩院,北京);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学);School of Integrated Circuits, Peking University(集成电路系,北京大学);YanXin MicroElectronics Co., Ltd.(YXME), Shanghai, China(燕芯微电子有限公司(YXME),上海,中国)
GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));UC Santa Cruz(加州大学圣克鲁兹分校);UIUC(伊利诺伊大学香槟分校);Southeast University(东南大学);BIAI, ZJUT(脑科学与类脑智能研究院,浙江大学);D5Data.ai
Dual-Space Smoothness for Robust and Balanced LLM Unlearning
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Department of Computer Science and Engineering, University of Notre Dame(圣母大学计算机科学与工程系)
Dissecting Fine-Tuning Unlearning in Large Language Models
International Digital Economy Academy (IDEA)(国际数字经济学院(IDEA));South China University of Technology(华南理工大学);Columbia University(哥伦比亚大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学)
From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Models
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Shanghai University of Electric Power(上海电力大学);School of Information Science and Engineering, East China University of Science and Technology(东华大学信息科学与工程学院)
Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models
Ludwig Maximilian University of Munich(慕尼黑路德维希-马克西米利安大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Aalto University(艾尔沃斯大学);Intel Labs(英特尔实验室)
Rethinking Machine Unlearning for Large Language Models
Michigan State University(密歇根州立大学);IBM Research(IBM研究院);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance Research(字节跳动研究院);Stanford University(斯坦福大学)
Attention Smoothing Is All You Need For Unlearning
Department of Computer Science, Wayne State University(韦恩州立大学计算机科学系);Department of Computer Science and Engineering, Michigan State University(密歇根州立大学计算机科学与工程系);Institute for AI and Data Science, Wayne State University(韦恩州立大学人工智能与数据科学研究所)
The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models
Institute for the Advancement of Legal and Ethical AI (ALEA Institute)(法律与伦理AI推进研究所(ALEA研究所));CodeX—The Stanford Center for Legal Informatics(CodeX——斯坦福法律信息学中心);Illinois Tech—Chicago Kent College of Law(伊利诺伊理工大学芝加哥肯特法学院);Center for Legal Technology & Data Science, Bucerius Law School(布塞留斯法学院法律技术与数据科学中心)
LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Technical University of Munich(慕尼黑技术大学);Siemens AG(西门子股份公司);University of Oxford(牛津大学)
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)