Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preserving its behavior on all other inputs. This process typically involves two stages: identifying the layer to edit and performing the parameter update. Intuitively, different queries may localize knowledge at different depths of the model, resulting in different sample-wise editing performance for a fixed editing layer. In this work, we hypothesize the existence of fixed golden layers that can achieve near-optimal editing performance similar to sample-wise optimal layers. To validate this hypothesis, we provide empirical evidence by comparing golden layers against ground-truth sample-wise optimal layers. Furthermore, we show that golden layers can be reliably identified using a proxy dataset and generalize effectively to unseen test set queries across datasets. Finally, we propose a novel method, namely Layer Gradient Analysis (LGA) that estimates golden layers efficiently via gradient-attribution, avoiding extensive trial-and-error across multiple editing runs. Extensive experiments on several benchmark datasets demonstrate the effectiveness and robustness of our LGA approach across different LLM types and various knowledge editing methods.
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany(马克斯·普朗克研究所信息学院,萨尔兰信息学院,德国萨尔布吕肯);Department of Language Science and Technology, Saarland University, Saarbrücken, Germany(语言科学与技术系,萨尔兰大学,德国萨尔布吕肯)
MILA - Quebec AI Institute(MILA - 魁北克人工智能研究所);Microsoft(微软公司);UNC Chapel Hill(北卡罗来纳大学教堂山分校);North Carolina State University(北卡罗来纳州立大学);Massachusetts Institute of Technology(麻省理工学院);Google DeepMind(谷歌DeepMind)
StruEdit: Structured Outputs Enable the Fast and Accurate Knowledge Editing for Large Language Models
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);University of Science and Technology of China(中国科学技术大学)
Improved approximation algorithms for capacitated cluster editing
Hangzhou Dianzi University(杭州电子科技大学);Zhejiang University of Water Resources and Electric Power(浙江水利水电学院);University of Alberta(阿尔伯塔大学);City University of Hong Kong(香港城市大学)