Parameter-based knowledge editing updates the internal knowledge of large language models (LLMs) via localized weight modifications and has attracted significant attention. However, most existing methods overlook fundamental theoretical limitations and are rarely evaluated under realistic, practice-oriented settings. In this paper, we first present a theoretical analysis based on the dimensional Collapse Hypothesis, explaining how localized parameter edits can propagate along fragile directions in the representation space, inducing global interference and ultimately causing reasoning collapse. Building on this insight, we conduct a comprehensive empirical evaluation by systematically varying knowledge complexity, number of edits, evaluation dimensions, and baseline methods. Our results show that parameter-based editing methods consistently damage core LLM capabilities. In contrast, a simple retrieval-based baseline achieves consistently stronger performance than all parameter-editing methods across all evaluated conditions. These findings highlight that preserving the fundamental capabilities of LLMs after knowledge editing should be a central concern for future research.
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
Bosch Center for Artificial Intelligence (BCAI)(博世人工智能中心(BCAI));Bosch (China) Investment Ltd.(博世(中国)投资有限公司);School of Statistics, East China Normal University(东华大学统计学院)
Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA(多模态人工智能系统国家重点实验室,中科院自动化所);School of Artificial Intelligence, UCAS(中国科学技术大学人工智能学院);Fudan University(复旦大学);Independent Researcher(独立研究者)
Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
Department of Computer Science and Engineering, POSTECH(POSTECH计算机科学与工程系);Designovel Co., Ltd.(Designovel公司);LLSOLLU;Graduate School of Artificial Intelligence, POSTECH(POSTECH人工智能研究生院)
Beijing University of Posts and Telecommunications(北京邮电大学);University of Science and Technology of China(中国科学技术大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);National University of Singapore(新加坡国立大学)
Department of Computer Science and Information Engineering(计算机科学与信息工程系);National Taiwan University(国立台湾大学);Artificial Intelligence Research Center(人工智能研究中心);AIST Japan(日本AIST);Institute of Information Science Academia Sinica(学术院信息科学研究所)
Generalizable Model Editing Could Be More Than Direct Generalization
University of Cambridge(剑桥大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of the Chinese Academy of Sciences(中国科学院大学);HUST(华中科技大学);BUPT(北京邮电大学);Shandong University(山东大学)
On the Robustness of Editing Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海市Web3可信数据流通与治理重点实验室)
Energy-Regularized Sequential Model Editing on Hyperspheres
Columbia University(哥伦比亚大学);University of California, Los Angeles(加州大学洛杉矶分校);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学)
Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Tsinghua University(清华大学);University of Electronic Science and Technology of China(电子科技大学)
FedAlphaEdit: Null-Space-Aligned Merging for Collaborative Knowledge Editing
Graduate School of Science and Technology, University of Tsukuba(筑波大学科学技术研究生院);Center for Artificial Intelligence Research, Tsukuba Institute for Advanced Research, University of Tsukuba(筑波大学筑波高级研究院人工智能研究中心);Institute of Systems and Information Engineering, University of Tsukuba(筑波大学系统与信息工程研究院)
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
The University of Hong Kong(香港大学);Tencent AI Lab(腾讯人工智能实验室);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);East China Normal University(华东师范大学)