LLM unlearning is essential for mitigating safety, copyright, and privacy concerns in pre-trained large language models (LLMs). Compared to preference alignment, it offers a more explicit way by removing undesirable knowledge characterized by specific unlearning datasets. In previous works, gradient ascent (GA) and its variants have shown promise for implementing unlearning, yet their untargeted nature results in unintended degradation of general capabilities, incomplete removal of knowledge, and the generation of incoherent responses, among many others. We argue that these issues stem from the absence of explicit guidance on what and how models should unlearn. To fill this gap, we introduce a novel unlearning target, reasoning-based unlearning target, which satisfies both the specified unlearning scope and the specified post-unlearning response. Building on this, we propose targeted reasoning unlearning (TRU), which leverages reasoning-based unlearning target as guidance. We employ the target using a cross-entropy supervised loss combined with a GA-based loss, enabling the model to learn reasoning ability for precise knowledge removal while preserving unrelated abilities. We evaluate TRU against strong baselines across multiple benchmarks and LLM backbones, and find that it achieves more reliable unlearning while preserving general capabilities. Moreover, TRU exhibits superior robustness under diverse attack scenarios, stemming from the reasoning ability learned through reasoning-based targets. Overall, our study establishes reasoning-augmented unlearning as a practical paradigm for reliable and explainable LLM unlearning.
Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(Mohamed bin Zayed人工智能大学);University of Queensland(昆士兰大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可证明责任人工智能与数据分析实验室);King Abdullah University of Science and Technology (KAUST)(卡塔尔科学与技术大学)
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
CURE:Circuit-Aware Unlearning for LLM-based Recommendation
University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学)
Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Models
School of Computing(计算机学院);National University of Singapore(新加坡国立大学);Department of Electrical and Computer Engineering(电气与计算机工程系);College of Design and Engineering(设计与工程学院);RespAI Lab(RespAI实验室);School of Computer Engineering(计算机工程学院);Institute for Infocomm Research(信息与通信研究所);A*STAR
A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
University of Information Technology (UIT)(信息科技大学);Ho Chi Minh City University of Technology (HCMUT)(胡志明市理工大学);Vietnam National University, Ho Chi Minh City(越南国家大学胡志明市分校);University of Economics Ho Chi Minh City (UEH)(胡志明市经济大学)
Towards Explainable Neural-Symbolic Visual Reasoning
Segula Technologies(赛古拉技术公司);Institut des Systèmes Intelligents et de Robotique, Sorbonne Universite(索邦大学智能系统与机器人研究所);INRIA(法国国家信息与自动化研究所);ENSTA Paris(巴黎高等先进技术学院)
许多高性能模型都缺乏可解释性。有越来越多的工作涌入可解释的人工智能(XAI),以解开什么是XAI的意思和期望。然而,对于如何产生和判断解释,并没有普遍的共识。在本文中,我们讨论了为什么整合连接主义和符号范式的技术是为非技术用户提供解释的最有效解决方案,并且我们提出了一个基于Doran et al. [2017](arXiv:1710.00794)定义的推理模型来解释神经网络的决策。我们使用这种解释来纠正网络决策原理中的偏差。我们根据Burns等人[2018](arXiv:1803.09797)中的图像描述方法,为该模型提供了一个潜在用途的示例。
Self-Exploring Language Models for Explainable Link Forecasting on Temporal Graphs via Reinforcement Learning
University of Cambridge(剑桥大学);Mila - Quebec AI Institute(魁北克人工智能研究院);McGill University(麦吉尔大学);University of Oxford(牛津大学);National University of Singapore(新加坡国立大学);University of Stuttgart(斯图加特大学);Amazon(亚马逊公司);AITHYRA
RES-MR: Risk-Aware Reasoning for Explainable and Safe Medication Recommendation
Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算机科学中心(国家济南超级计算中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心);University of Technology Sydney(悉尼科技大学)
Knowledge-Augmented Large Language Model Agents for Explainable Financial Decision-Making
Boston University(波士顿大学);Hofstra University(霍夫斯特拉大学);University of Southern California(南加州大学);Georgia Institute of Technology(佐治亚理工学院);Carnegie Mellon University(卡内基梅隆大学)
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore(新加坡高性能计算研究所,科技研究局);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(科技研究局前沿人工智能研究中心,新加坡);College of Computing and Data Science, Nanyang Technological University, Singapore(新加坡南洋理工大学计算与数据科学学院)
EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(电子与电气工程系,香港理工大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电子与电气工程学院,南洋理工大学);Shien-Ming Wu School of Intelligent Engineering, South China University of Technology(智能工程学院,华南理工大学)