Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Peking University（北京大学）；Provable Responsible AI and Data Analytics (PRADA) Lab（可验证负责任人工智能与数据分析（PRADA）实验室）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；South China University of Technology（华南理工大学）
Zhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng, Lijie Hu, Di Wang
The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recall tasks, they consistently struggle with multi-hop factual recall tasks involving newly edited knowledge. In this paper, leveraging tools in mechanistic interpretability, we first identify that in multi-hop tasks, LLMs tend to retrieve knowledge with implicit subject information from deeper MLP layers, unlike single-hop tasks, which rely on shallow layers. This distinction explains the poor performance of current methods in multi-hop queries, as they primarily focus on editing shallow layers with single-hop edit prompts, leaving deeper layers unchanged. To address this, we propose IFMET, a novel locate-then-edit KE approach designed to edit both shallow and deep MLP layers. Beyond single-hop editing prompts, IFMET further incorporates multi-hop editing prompts to locate and modify knowledge across different stages of reasoning. Experimental results demonstrate that IFMET significantly improves performance on multi-hop factual recall tasks, overcoming the limitations of previous locate-then-edit methods
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
University of Amsterdam（阿姆斯特丹大学）；University of Edinburgh（爱丁堡大学）
The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive re-training or fine-tuning. Besides being computationally efficient, KnowledgeEditordoes not require any modifications in LM pre-training (e.g., the use of meta-learning). In our…
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
University of Science and Technology of China（中国科学技术大学）；City University of Hong Kong（香港城市大学）；Jarvis Research Center（贾维斯研究中心）；Tencent YouTu Lab（腾讯优图实验室）；Peking University（北京大学）
ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall
HKUST(GZ) Deep Interdisciplinary Intelligence Lab（香港科技大学(广州)深 interdisciplinary 智能实验室）；BUAA（北京航空航天大学）；HKUST(GZ) Institute of Deep Perception Technology, JITRI Deep Interdisciplinary Intelligence Lab（香港科技大学(广州)深度感知技术研究所，JITRI 深 interdisciplinary 智能实验室）
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；School of Cyberspace Security, Southeast University（东南大学网络空间安全学院）
Factual and Edit-Sensitive Graph-to-Sequence Generation via Graph-Aware Adaptive Noising
Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi, India（印度理工学院德里分校电子工程系）；Indian Institute of Science, Bengaluru, India（印度科学研究所，班加罗尔，印度）
Assessing Knowledge Editing in Language Models via Relation Perspective
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, CAS（中国科学院自动化研究所）；Kunming University of Science and Technology（昆明理工大学）；Beijing Institute of Technology（北京理工大学）
Seoul National University（首尔国立大学）；AIRS Company, Hyundai Motor Group（现代汽车集团AIRS公司）；Adobe Research（奥多比研究院）
Despite the recent advancements in abstractive summarization systems leveraged from large-scale datasets and pre-trained language models, the factual correctness of the summary is still insufficient. One line of trials to mitigate this problem is to include a post-editing process that can detect and correct factual errors in the summary. In building such a post-editing system, it is strongly required that 1) the process has a high success rate and interpretability and 2) has a fast running time. Previous approaches focus on regeneration of the summary using the autoregressive models, which lac…
REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing
NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所国家信息处理实验室、人工智能研究所）；Cuiying Honors College, Lanzhou University（兰州大学萃英荣誉学院）；Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）；Tencent（腾讯）
KEditVis: A Visual Analytics System for Knowledge Editing of Large Language Models
Zhejiang University（浙江大学）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；Hong Kong University of Science and Technology（香港科技大学）；School of Software Engineering, South China University of Technology（华南理工大学软件学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）
Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of California, Merced（加州大学默塞德分校）；University of Science and Technology of China（中国科学技术大学）
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
School of Computer Science and Technology, Tianjin University（天津大学计算机科学与技术学院）；Kexin Technology（可心科技）；University of Chinese Academy of Sciences（中国科学院大学）；Tencent Hunyuan（腾讯混元）；College of Computer Science, Chongqing University（重庆大学计算机学院）
WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Tübingen AI Center, University of Tübingen（图宾根人工智能中心，图宾根大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Technical University, Munich（慕尼黑技术大学）；University of Virginia（弗吉尼亚大学）
Surgical Knowledge Rewrite in Compact LLMs: An 'Unlearn-then-Learn' Strategy with ($IA^3$) for Localized Factual Modulation and Catastrophic Forgetting Mitigation
EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
The Hong Kong Polytechnic University（香港理工大学）；Hangzhou Diagens Biotechnology Co., Ltd（杭州迪安基因生物技术有限公司）；University of Science and Technology of China（中国科学技术大学）
Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Tsinghua University（清华大学）；University of Electronic Science and Technology of China（电子科技大学）
EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing
Institute for Clarity in Documentation（文档清晰度研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康库尔中心）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕默研究实验室）
