arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.11263cs.CLcs.AI
收藏

理解大语言模型在模型编辑中的崩溃现象

Understanding the Collapse of LLMs in Model Editing

Institute of Computing Technology, CAS(中国科学院计算技术研究所);Baidu Inc.(百度公司);University of Chinese Academy of Sciences(中国科学院大学)

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Du Su, Dawei Yin, Huawei Shen

AI总结:

本文通过分析发现模型编辑中崩溃源于带前缀与不带前缀键处理不一致及首 token 键分布差异,提出统一使用带前缀键并保持训练测试一致的方法,有效防止 LLM 崩溃并维持编辑效果。

摘要:

尽管模型编辑方法取得了显著进展,但其在真实场景中的应用仍然具有挑战性,因为它们经常导致大语言模型(LLMs)崩溃。其中,ROME 尤其令人担忧,因为仅一次编辑就可能破坏 LLMs。本文研究了这种崩溃的根本原因。通过广泛分析,我们确定了导致崩溃的两个主要因素:i)在参数更新方程中对带前缀和不带前缀的键的不一致处理可能导致非常小的分母,从而引起过大的参数更新;ii)崩溃案例的主语通常是第一个 token,在自回归 transformer 中,其不带前缀的键分布与带前缀的键分布显著不同,导致上述问题实际发生。为验证我们的发现,我们提出了一种简单而有效的方法:在编辑阶段统一使用带前缀的键,并在测试阶段添加前缀,以确保训练和测试之间的一致性。实验结果表明,所提出的解决方案能够防止模型崩溃,同时保持编辑的有效性。

英文摘要:

Despite significant progress in model editing methods, their application in real-world scenarios remains challenging as they often cause large language models (LLMs) to collapse. Among them, ROME is particularly concerning, as it could disrupt LLMs with only a single edit. In this paper, we study the root causes of such collapse. Through extensive analysis, we identify two primary factors that contribute to the collapse: i) inconsistent handling of prefixed and unprefixed keys in the parameter update equation may result in very small denominators, causing excessively large parameter updates; ii) the subject of collapse cases is usually the first token, whose unprefixed key distribution significantly differs from the prefixed key distribution in autoregressive transformers, causing the aforementioned issue to materialize. To validate our findings, we propose a simple yet effective approach: uniformly using prefixed keys during editing phase and adding prefixes during testing phase to ensure the consistency between training and testing. The experimental results show that the proposed solution can prevent model collapse while maintaining the effectiveness of the edits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑