InstructEdit：基于指令的大语言模型知识编辑
InstructEdit: Instruction-based Knowledge Editing for Large Language Models
AI总结：
针对大语言模型知识编辑跨任务泛化性受限的问题，提出基于指令的统一编辑技术InstructEdit，在多任务设置下显著提升了编辑器的控制能力与泛化表现。
摘要：
大语言模型的知识编辑可以提供一种高效的解决方案，在不负面影响整体性能的情况下改变模型的行为。然而，当前的方法面临跨任务泛化性有限的问题，每个任务需要一个独立的编辑器，这严重阻碍了更广泛的应用。为了解决这个问题，我们率先分析了知识编辑中的多任务泛化问题。具体而言，我们开发了一种基于指令的编辑技术，称为InstructEdit，它利用简单的指令使编辑器能够同时适应各种任务的性能。每个LLM仅使用一个统一的编辑器，我们通过实验证明，InstructEdit能够提升对编辑器的控制，在多任务编辑设置中，Reliability平均提升14.86%。此外，涉及保留未见任务的实验表明，InstructEdit持续超越先前的强基线。为了进一步研究基于指令的知识编辑的潜在机制，我们分析了编辑梯度方向的主成分，这表明指令有助于控制优化方向，并具有更强的OOD泛化能力。代码和数据集可在https://github.com/zjunlp/EasyEdit获取。
英文摘要：
Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approaches encounter issues with limited generalizability across tasks, necessitating one distinct editor for each task, significantly hindering the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets are available in https://github.com/zjunlp/EasyEdit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
