Neural language models (LMs) represent facts about the world described by text. Sometimes these facts derive from training data (in most LMs, a representation of the word "banana" encodes the fact that bananas are fruits). Sometimes facts derive from input text itself (a representation of the sentence "I poured out the bottle" encodes the fact that the bottle became empty). We describe REMEDI, a method for learning to map statements in natural language to fact encodings in an LM's internal representation system. REMEDI encodings can be used as knowledge editors: when added to LM hidden representations, they modify downstream generation to be consistent with new facts. REMEDI encodings may also be used as probes: when compared to LM representations, they reveal which properties LMs already attribute to mentioned entities, in some cases making it possible to predict when LMs will generate outputs that conflict with background knowledge or input text. REMEDI thus links work on probing, prompting, and LM editing, and offers steps toward general tools for fine-grained inspection and control of knowledge in LMs.
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Knowledge Graph Enhanced Large Language Model Editing
School of Computer Science and Technology, Shandong University（山东大学计算机科学与技术学院）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）
MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge
BIGAI（北京智源人工智能研究院）；Shandong University（山东大学）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Beijing Institute of Technology（北京理工大学）
Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Artificial Intelligence Innovation Center, Yangtze Delta Region Institute of Tsinghua University, Zhejiang（浙江清华长三角研究院人工智能创新中心）；Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University（东北大学医学影像智能计算教育部重点实验室）
History Matters: Temporal Knowledge Editing in Large Language Model
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Center for Data Science, Peking University（北京大学数据科学中心）；School of Law, Tsinghua University（清华大学法学院）
Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editing
School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）；Department of Data Science and AI, Monash University（莫纳什大学数据科学与人工智能系）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Department of Computer Science, Emory University（埃默里大学计算机科学系）
MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editing
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所认知与决策智能复杂系统实验室）
Department of Computer Science and Information Engineering（计算机科学与信息工程系）；National Taiwan University（国立台湾大学）；Artificial Intelligence Research Center（人工智能研究中心）；AIST Japan（日本AIST）；Institute of Information Science Academia Sinica（学术院信息科学研究所）
Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；University of Electronic Science and Technology of China（电子科技大学）
Uncovering Overfitting in Large Language Model Editing
Shandong University（山东大学）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
SetKE: Knowledge Editing for Knowledge Elements Overlap
State Key Laboratory of CCSE,School of Computer Science and Engineering,Beihang University（信息与通信工程国家重点实验室，计算机科学与工程学院，北航）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing Institute of Technology（北京理工大学）；Kunming University of Science and Technology（昆明理工大学）
Energy-Regularized Sequential Model Editing on Hyperspheres
Columbia University（哥伦比亚大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Zhejiang University（浙江大学）；University of Science and Technology of China（中国科学技术大学）
Disentangling Knowledge Representations for Large Language Model Editing
Shandong University（山东大学）；Suzhou Research Institute of Shandong University（山东大学苏州研究院）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences（模式识别新实验室（NLPR）多模态人工智能系统国家重点实验室（MAIS）中国科学院自动化研究所）；Leiden University（莱顿大学）
Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
