Efficient knowledge editing of large language models is crucial for replacing obsolete information or incorporating specialized knowledge on a large scale. However, previous methods implicitly assume that knowledge is localized and isolated within the model, an assumption that oversimplifies the interconnected nature of model knowledge. The premise of localization results in an incomplete knowledge editing, whereas an isolated assumption may impair both other knowledge and general abilities. It introduces instability to the performance of the knowledge editing method. To transcend these assumptions, we introduce StableKE, a method adopts a novel perspective based on knowledge augmentation rather than knowledge localization. To overcome the expense of human labeling, StableKE integrates two automated knowledge augmentation strategies: Semantic Paraphrase Enhancement strategy, which diversifies knowledge descriptions to facilitate the teaching of new information to the model, and Contextual Description Enrichment strategy, expanding the surrounding knowledge to prevent the forgetting of related information. StableKE surpasses other knowledge editing methods, demonstrating stability both edited knowledge and multi-hop knowledge, while also preserving unrelated knowledge and general abilities. Moreover, StableKE can edit knowledge on ChatGPT.
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Knowledge Graph Enhanced Large Language Model Editing
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院)
MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge
BIGAI(北京智源人工智能研究院);Shandong University(山东大学);University of Science and Technology of China(中国科学技术大学);Peking University(北京大学);Beijing Institute of Technology(北京理工大学)
Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Artificial Intelligence Innovation Center, Yangtze Delta Region Institute of Tsinghua University, Zhejiang(浙江清华长三角研究院人工智能创新中心);Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University(东北大学医学影像智能计算教育部重点实验室)
History Matters: Temporal Knowledge Editing in Large Language Model
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Center for Data Science, Peking University(北京大学数据科学中心);School of Law, Tsinghua University(清华大学法学院)
Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editing
School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院);Department of Data Science and AI, Monash University(莫纳什大学数据科学与人工智能系);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Department of Computer Science, Emory University(埃默里大学计算机科学系)
LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editing
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Department of Computer Science and Information Engineering(计算机科学与信息工程系);National Taiwan University(国立台湾大学);Artificial Intelligence Research Center(人工智能研究中心);AIST Japan(日本AIST);Institute of Information Science Academia Sinica(学术院信息科学研究所)
Uncovering Overfitting in Large Language Model Editing
Shandong University(山东大学);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
SetKE: Knowledge Editing for Knowledge Elements Overlap
State Key Laboratory of CCSE,School of Computer Science and Engineering,Beihang University(信息与通信工程国家重点实验室,计算机科学与工程学院,北航);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Beijing Institute of Technology(北京理工大学);Kunming University of Science and Technology(昆明理工大学)
Energy-Regularized Sequential Model Editing on Hyperspheres
Columbia University(哥伦比亚大学);University of California, Los Angeles(加州大学洛杉矶分校);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学)
Disentangling Knowledge Representations for Large Language Model Editing
Shandong University(山东大学);Suzhou Research Institute of Shandong University(山东大学苏州研究院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences(模式识别新实验室(NLPR)多模态人工智能系统国家重点实验室(MAIS)中国科学院自动化研究所);Leiden University(莱顿大学)
Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
In-Context Editing: Learning Knowledge from Self-Induced Distributions
State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,北京智源人工智能研究院);University of Science and Technology of China(中国科学技术大学);Peking University(北京大学)
Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
Key Lab of Intell. Info. Process., Inst. of Comput. Tech., CAS(智能信息处理重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Agriculture Information Institute, CAAS(农业信息研究所,中国农业科学院)
Related Knowledge Perturbation Matters: Rethinking Multiple Pieces of Knowledge Editing in Same-Subject
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
EtCon: Edit-then-Consolidate for Reliable Knowledge Editing
Wuhan University(武汉大学);Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学)
National Engineering Research Center of Speech and Language Information Processing(语音与语言信息处理国家级工程研究中心);University of Science and Technology of China(中国科学技术大学);iFLYTEK Research(iFLYTEK研究院);University of California, Los Angeles(美国加州大学洛杉矶分校)
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
School of Computer Science and Technology, Tianjin University(天津大学计算机科学与技术学院);Kexin Technology(可心科技);University of Chinese Academy of Sciences(中国科学院大学);Tencent Hunyuan(腾讯混元);College of Computer Science, Chongqing University(重庆大学计算机学院)
Assessing Knowledge Editing in Language Models via Relation Perspective
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, CAS(中国科学院自动化研究所);Kunming University of Science and Technology(昆明理工大学);Beijing Institute of Technology(北京理工大学)