Keys to Robust Edits: from Theoretical Insights to Practical Advances
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖先进研究院技术研究所);Shanghai AI Lab(上海人工智能实验室)
Large language models (LLMs) struggle with maintaining accurate knowledge due to conflicting/outdated parametric memories. While locate-and-edit methods address this, their reliance on models' internal representations leads to robustness failures in long-context reasoning and paraphrased queries. We identify a fundamental limitation of locate-and-edit methods: existing semantic keys (for memory localization) cannot simultaneously satisfy robustness (context-invariant activation) and specificity (precise knowledge discrimination). Through theoretical error-bound analysis, we establish formal criteria for effective editing. Our solution introduces \textit{Robust Edit Pathway (REP)}, a plug-and-play module that: (1) disentangles editing keys from native model representations; (2) dynamically adjusts keys via contrastive learning to achieve robustness-specificity balance. Extensive experiments across various editing methods (ROME/MEMIT/R-ROME/EMMET), existing LLMs (LLaMA2, QWen, Mistral), and datasets (CounterFact, ZsRE) show that REP improves success rate over robustness tests by up-to 66.4\% while maintaining the success rate unaffected. Our code can be found at https://github.com/ElliottYan/RobustKeyEdit .
Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers
CSE, HKUST(香港科技大学计算机科学与工程系);IIIS, Tsinghua University(清华大学交叉信息研究院);ILCC, University of Edinburgh(爱丁堡大学语言、语言学与文化学院);College of Computer Science, Sichuan University(四川大学计算机学院)
Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs
MoE Key Laboratory of Interdisciplinary Research of Computation and Economics(交叉计算与经济学 interdisciplinary 研究联合实验室);Shanghai University of Finance and Economics(上海财经大学)
CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
Multimodal Art Projection Research Community(多模态艺术投影研究社区);University of Waterloo(滑铁卢大学);HKUST(香港科技大学);University of Manchester(曼彻斯特大学);Tongji University(同济大学);Vector Institute(矢量研究所)
LLMs Meet Multimodal Generation and Editing: A Survey
The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室)
ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systems
Zhejiang University,\ University - Ant \ Joint Laboratory of Knowledge Graph Hangzhou China;Zhejiang University,\ University - Ant \ Joint Laboratory of Knowledge Graph