发表机构
Yale University; Washington University in St. Louis; Icahn School of Medicine at Mount Sinai; Arizona State University（耶鲁大学; 圣路易斯华盛顿大学; 西奈山伊坎医学院; 亚利桑那州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
LadderEdit通过阶梯式提升LoRA适配器秩的压缩方法，在维持编辑覆盖范围的同时，将LLM终身编辑的存储需求降低5.2倍，且在5万次连续编辑下仍保持有效性。
AI 中文摘要
大语言模型（LLM）的终身编辑需要在获取编辑后存储数千次编辑。一类广泛使用的方法是为每次编辑附加一个LoRA适配器，该方法能保留模型行为，但存储量呈线性增长。为解决这一挑战，我们提出LadderEdit，一种在每次编辑获取后对LoRA适配器进行压缩的方法。每次编辑首先以低秩形式存储为低成本草图，随后检查该草图是否仍满足探测提示上的重写、泛化和局部性约束。通过检查的编辑保留该草图，未通过的编辑则沿“阶梯”提升至更高秩，直至满足约束。由于每次编辑都保留了部分表示，覆盖范围得以维持，仅困难编辑会消耗更多秩。在LLaMA-3-8B、Mistral-7B和Qwen2.5-7B上的ZsRE、CounterFact和WikiBigEdit基准测试中，LadderEdit的存储量仅为精确LoRA的1/5.2，且在50000次连续编辑下仍保持有效。
英文摘要
Lifelong editing of LLMs requires storing thousands of edits after acquisition. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired. Each edit is first stored at low rank as a cheap sketch. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.
CommentsEMNLP 2026 Main Conference Long Paper
