Despite significant progress in model editing methods, their application in real-world scenarios remains challenging as they often cause large language models (LLMs) to collapse. Among them, ROME is particularly concerning, as it could disrupt LLMs with only a single edit. In this paper, we study the root causes of such collapse. Through extensive analysis, we identify two primary factors that contribute to the collapse: i) inconsistent handling of prefixed and unprefixed keys in the parameter update equation may result in very small denominators, causing excessively large parameter updates; ii) the subject of collapse cases is usually the first token, whose unprefixed key distribution significantly differs from the prefixed key distribution in autoregressive transformers, causing the aforementioned issue to materialize. To validate our findings, we propose a simple yet effective approach: uniformly using prefixed keys during editing phase and adding prefixes during testing phase to ensure the consistency between training and testing. The experimental results show that the proposed solution can prevent model collapse while maintaining the effectiveness of the edits.
Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse
Shandong University(山东大学);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
School of Computer Science and Technology, Tianjin University(天津大学计算机科学与技术学院);Kexin Technology(可心科技);University of Chinese Academy of Sciences(中国科学院大学);Tencent Hunyuan(腾讯混元);College of Computer Science, Chongqing University(重庆大学计算机学院)
The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse
CAS Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Baidu Inc.(百度公司)
MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
City University of Hong Kong(香港城市大学);Harvard Medical School(哈佛医学院);Institute of Automation, CAS(中国科学院自动化研究所);Columbia University(哥伦比亚大学);Ohio State University(俄亥俄州立大学);Westlake University(西湖大学);Nanjing University(南京大学)
Commonsense Knowledge Editing Based on Free-Text in LLMs
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)
NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database
Department of Mathematical Sciences, Tsinghua University(清华大学数学科学系);IIIS, Tsinghua University(清华大学人工智能学院);College of AI, Tsinghua University(清华大学人工智能学院);Huawei Technologies Co., Ltd.(华为技术有限公司)
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Jarvis Research Center(贾维斯研究中心);Tencent YouTu Lab(腾讯优图实验室);Peking University(北京大学)
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
Constraining Sequential Model Editing with Editing Anchor Compression
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China(中国科学技术大学语音及语言信息处理国家工程研究中心);Zhejiang University(浙江大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
Bayesian Optimization for Controlled Image Editing via LLMs
University of Edinburgh(爱丁堡大学);University of Manchester(曼彻斯特大学);University of Washington(华盛顿大学);Tsinghua University(清华大学);Skai Intelligence(Skai智能科技);University of Copenhagen(哥本哈根大学)
Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning
School of Economics and Management, East China Normal University(华东师范大学经济与管理学部);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Knowledge Graph Enhanced Large Language Model Editing
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院)
Potential and Challenges of Model Editing for Social Debiasing
Zhejiang University School of Engineering(浙江大学工程学院);Westlake University(西湖大学);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究所)
MEGen: Generative Backdoor into Large Language Models via Model Editing
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University(上海交通大学智能交互与认知工程重点实验室);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海Web3可信数据流通与治理重点实验室);Cognitive AI Lab(认知人工智能实验室)
CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
Multimodal Art Projection Research Community(多模态艺术投影研究社区);University of Waterloo(滑铁卢大学);HKUST(香港科技大学);University of Manchester(曼彻斯特大学);Tongji University(同济大学);Vector Institute(矢量研究所)
Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
Key Lab of Intell. Info. Process., Inst. of Comput. Tech., CAS(智能信息处理重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Agriculture Information Institute, CAAS(农业信息研究所,中国农业科学院)
Unveiling the Pitfalls of Knowledge Editing for Large Language Models
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);ZJU-Ant Group Joint Research Center for Knowledge Graphs, Zhejiang University(浙江大学-蚂蚁集团知识图谱联合研究中心);ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University(浙江大学杭州国际科技创新中心);Tencent(腾讯)
Editing Conceptual Knowledge for Large Language Models
Zhejiang University(浙江大学);ZJU-Ant Group Joint Research Center for Knowledge Graphs(浙江大学-蚂蚁集团知识图谱联合研究中心);National University of Singapore(新加坡国立大学);Ant Group(蚂蚁集团)
Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)