As large language models (LLMs) are increasingly deployed in real-world systems, they must support post-hoc removal of specific content to meet privacy and governance requirements. This motivates selective unlearning, which suppresses information about a particular entity or topic while preserving the LLM's general utility. However, most existing LLM unlearning methods require access to the original training corpus and rely on output-level refusal tuning or broad gradient updates, creating a tension among unlearning strength, non-target preservation, and data availability. We propose Geometric Unlearning (GU), an approach that operates directly on the model's prompt-conditioned hidden states without access to the original training corpus. Specifically, GU distills a compact, low-rank safe-behavior subspace from a small set of safe reference prompts and uses lightweight anchor-in-context synthetic prompts to trigger localized, projection-based alignment of hidden representations to this safe subspace. A teacher-distillation regularizer on synthetic non-target anchors further reduces collateral drift. Across privacy-oriented unlearning benchmarks (ToFU and UnlearnPII), GU achieves strong target suppression with minimal impact on non-target performance, demonstrating that effective unlearning can be achieved with minimal synthetic data.
Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting
Faculty of Information Technology, Monash University(墨尔本大学信息科技学院);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(教育部计算能力网络与信息安全重点实验室,山东计算机科学中心,齐鲁大学(山东科学院));Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science(山东省计算能力互联网与服务计算重点实验室,山东省计算机科学基础研究中心);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院)
Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation
GSCST, Seoul National University(首尔国立大学GSCST);AIIS, Seoul National University(首尔国立大学AIIS);Department of Artificial Intelligence, Chung-Ang University(Chung-Ang大学人工智能系);Korean Surgical Researcher Foundation, Republic of Korea(韩国外科研究员基金会)
CRISP: Persistent Concept Unlearning via Sparse Autoencoders
Technion – Israel Institute of Technology(技术ion-以色列理工学院);Boston University(波士顿大学);University of Zagreb, FER(Zagreb大学,FER);Kempner Institute, Harvard University(哈佛大学 Kempner研究所)
Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
The Hong Kong Polytechnic University(香港理工大学);Carnegie Mellon University(卡内基梅隆大学);University of California, Santa Cruz(加州大学圣克ruz分校);Huawei Technologies(华为技术有限公司);Research Centre for Privacy and Security Technologies in Future Smart Systems, PolyU(未来智能系统中的隐私与安全技术研究中心,PolyU)
School of Advanced Interdisciplinary Sciences, UCAS(UCAS交叉学科研究院);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院);Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Rethinking Machine Unlearning for Large Language Models
Michigan State University(密歇根州立大学);IBM Research(IBM研究院);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of California, Santa Cruz(加州大学圣克鲁兹分校);ByteDance Research(字节跳动研究院);Stanford University(斯坦福大学)