We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model's factual predictions. This reveals a distinct set of steps in middle-layer feed-forward modules that mediate factual predictions while processing subject tokens. To test our hypothesis that these computations correspond to factual association recall, we modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another. Our results confirm an important role for mid-layer feed-forward modules in storing factual associations and suggest that direct manipulation of computational mechanisms may be a feasible approach for model editing. The code, dataset, visualizations, and an interactive demo notebook are available at https://rome.baulab.info/
Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);University of Electronic Science and Technology of China(电子科技大学)
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Peking University(北京大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可验证负责任人工智能与数据分析(PRADA)实验室);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);South China University of Technology(华南理工大学)
Reasoning-Oriented and Analogy-Based Methods for Locating and Editing in Zero-Shot Event-Relational Reasoning
School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院);School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);School of Computer Science and Artificial Intelligence, Liaoning Normal University(辽宁师范大学计算机与人工智能学院)
本文提出 ROLE 和 ABLE 两种定位与编辑方法,分别通过定位语言模型关键模块和利用任务间类比关系,高效提升零样本事件关系推理的可解释性与性能。
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Cyberspace Security, Southeast University(东南大学网络空间安全学院)
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall
HKUST(GZ) Deep Interdisciplinary Intelligence Lab(香港科技大学(广州)深 interdisciplinary 智能实验室);BUAA(北京航空航天大学);HKUST(GZ) Institute of Deep Perception Technology, JITRI Deep Interdisciplinary Intelligence Lab(香港科技大学(广州)深度感知技术研究所,JITRI 深 interdisciplinary 智能实验室)
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Jarvis Research Center(贾维斯研究中心);Tencent YouTu Lab(腾讯优图实验室);Peking University(北京大学)
Assessing Knowledge Editing in Language Models via Relation Perspective
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, CAS(中国科学院自动化研究所);Kunming University of Science and Technology(昆明理工大学);Beijing Institute of Technology(北京理工大学)
REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing
NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所国家信息处理实验室、人工智能研究所);Cuiying Honors College, Lanzhou University(兰州大学萃英荣誉学院);Department of Mathematics, The Chinese University of Hong Kong(香港中文大学数学系);Tencent(腾讯)
MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models
University of Science and Technology of China(中国科学技术大学);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,北京通用人工智能研究院);Northeast Forestry University(东北林业大学);C-FAIR&school of software, Shandong University(山东大学软件学院C-FAIR);State Key Lab. for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);The University of Sydney(悉尼大学);Anhui Polytechnic University(安徽工程大学);Ludwig Maximilian University of Munich(慕尼黑大学)