曾经学到的知识可能需要被遗忘：面向大语言模型中废弃API知识的机器遗忘

中文摘要 AI 辅助

用于代码补全的大型语言模型（LLMs）可能会生成已废弃的API，因为其预训练语料包含来自历史库版本的代码。现有方法采用推理时干预、模型编辑或机器遗忘，但多种合理补全使得预定义替换具有局限性。此外，现有研究很少验证模型是否表现出目标废弃行为，或评估对其他API的意外更改。我们对面向废弃API知识的机器遗忘进行了系统性实证研究，并构建了MUDAPIBench，这是一个基于行为的基准，包含超过7,000个模型特定实例，这些实例源自跨越八个Python库的145个废弃到最新API映射。仅当原始模型生成目标废弃API时，实例才被保留。我们评估了三种代码LLM上的八种代表性遗忘方法，涉及废弃API遗忘、最新API生成、其他及无关API行为保持、通用代码生成能力和效率。结果表明，梯度差异（GD）提供了最佳整体权衡，在抑制废弃API的同时以适中的计算成本保持其他能力。进一步分析揭示了库之间的显著差异，并表明在模型训练数据截止日期之后废弃的API更难被遗忘。分层分析表明，GD实现了有效遗忘，且内部变化相对受控。

英文摘要

Large language models (LLMs) for code completion may generate deprecated APIs because their pre-training corpora contain code from historical library versions. Existing approaches use inference-time intervention, model editing, or machine unlearning, but multiple plausible completions make predefined replacements restrictive. Moreover, existing studies rarely verify whether models exhibit the targeted deprecated behavior or evaluate unintended changes to other APIs. We conduct a systematic empirical study of machine unlearning for deprecated API knowledge and construct MUDAPIBench, a behavior-grounded benchmark with over 7,000 model-specific instances derived from 145 deprecated-to-up-to-date API mappings across eight Python libraries. Instances are retained only when the original model generates the target deprecated API. We evaluate eight representative unlearning methods across three code LLMs on deprecated API forgetting, up-to-date API generation, preservation of other and unrelated API behaviors, general code-generation capability, and efficiency. Results show that Gradient Difference (GD) provides the best overall trade-off, suppressing deprecated APIs while preserving other capabilities with moderate computational costs. Further analyses reveal substantial variation across libraries and show that APIs deprecated after the model's training-data cutoff are harder to forget. Layer-wise analyses indicate that GD achieves effective forgetting with comparatively controlled internal changes.

发表机构

• School of Computer Science, Wuhan University（武汉大学计算机学院）
• Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
• School of Information Technology, Deakin University（迪肯大学信息技术学院）
• The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）
• School of Science and Technology, Hong Kong Metropolitan University（香港都会大学理学院）

机构由 AI 辅助整理，请以论文原文为准。