使用规范示例进行模型编辑
Model Editing with Canonical Examples
AI总结:
本文提出一种模型编辑新设置,利用规范示例进行少样本学习,并引入语义微调方法,在Pythia和Backpack模型上验证了其有效性,显著提升编辑性能。
摘要:
我们引入了使用规范示例进行模型编辑的设置,其中(1)每个期望行为仅提供一个学习示例,(2)评估完全在分布外进行,并且(3)与初始模型的偏差受到严格限制。规范示例是良好行为的简单实例,例如“毛里求斯的首都是路易港”,或不良行为,例如“研究人员的一个方面是冷酷无情”。评估集包含每个行为的更复杂示例(如一段提到毛里求斯首都的段落)。我们创建了三个数据集并修改了另外三个数据集,用于规范示例的模型编辑,涵盖知识密集型改进、社会偏见缓解和句法边缘情况。在我们对Pythia语言模型的实验中,我们发现LoRA优于全量微调和MEMIT。然后,我们转向Backpack语言模型架构,因为它旨在实现有针对性的改进。Backpack定义了一个大型的语义向量库——对每个词不同用法的分解——这些向量被加权求和以形成模型的输出logits。我们提出了语义微调,即为每个规范示例选择并微调少量(约10个)语义向量,并发现它优于其他微调方法,例如4.8%的提升对比0.3%。最后,我们通过仅使用35倍更小的Backpack的语义微调变化进行推理时集成,改进了GPT-J-6B,在一种设置中优于直接编辑GPT-J本身(4.1%对比1.0%)。
英文摘要:
We introduce model editing with canonical examples, a setting in which (1) a single learning example is provided per desired behavior, (2) evaluation is performed exclusively out-of-distribution, and (3) deviation from an initial model is strictly limited. A canonical example is a simple instance of good behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g., An aspect of researchers is coldhearted). The evaluation set contains more complex examples of each behavior (like a paragraph in which the capital of Mauritius is called for.) We create three datasets and modify three more for model editing with canonical examples, covering knowledge-intensive improvements, social bias mitigation, and syntactic edge cases. In our experiments on Pythia language models, we find that LoRA outperforms full finetuning and MEMIT. We then turn to the Backpack language model architecture because it is intended to enable targeted improvement. The Backpack defines a large bank of sense vectors--a decomposition of the different uses of each word--which are weighted and summed to form the output logits of the model. We propose sense finetuning, which selects and finetunes a few ($\approx$ 10) sense vectors for each canonical example, and find that it outperforms other finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve GPT-J-6B by an inference-time ensemble with just the changes from sense finetuning of a 35x smaller Backpack, in one setting outperforming editing GPT-J itself (4.1% vs 1.0%).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。