使用任务算术编辑模型
Editing Models with Task Arithmetic
AI总结：
本文提出以 task vector 为核心的模型编辑新范式，通过权重相减构建任务向量，并利用取反、相加等算术运算有效引导模型在目标任务、多任务及类比任务上的行为表现。
摘要：
改变预训练模型的行为——例如，提升其在下游任务上的表现，或减轻在预训练中学习到的偏差——是开发机器学习系统时的常见做法。在这项工作中，我们提出了一种以 task vectors 为核心的引导神经网络行为的新范式。task vector 指定了预训练模型权重空间中的一个方向，使得沿该方向移动能提升该任务的表现。我们通过从预训练模型的权重中减去同一模型在某个任务上微调后的权重来构建 task vector。我们展示了这些 task vector 可以通过取反和相加等算术运算进行修改和组合，并且所得模型的行为也会相应地被引导。对 task vector 取反会降低目标任务的表现，而对控制任务上的模型行为几乎没有影响。此外，将多个 task vector 相加可以同时提升多个任务的表现。最后，当任务之间以“A之于B犹如C之于D”的类比关系相联系时，组合其中三个任务的 task vector 可以提升第四个任务的表现，即使没有使用第四个任务的任何数据进行训练。总体而言，我们在多个模型、模态和任务上的实验表明，task arithmetic 是一种简单、高效且有效的模型编辑方法。
英文摘要：
Changing how pre-trained models behave -- e.g., improving their performance on a downstream task or mitigating biases learned during pre-training -- is a common practice when developing machine learning systems. In this work, we propose a new paradigm for steering the behavior of neural networks, centered around \textit{task vectors}. A task vector specifies a direction in the weight space of a pre-trained model, such that movement in that direction improves performance on the task. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly. Negating a task vector decreases performance on the target task, with little change in model behavior on control tasks. Moreover, adding task vectors together can improve performance on multiple tasks at once. Finally, when tasks are linked by an analogy relationship of the form ``A is to B as C is to D", combining task vectors from three of the tasks can improve performance on the fourth, even when no data from the fourth task is used for training. Overall, our experiments with several models, modalities and tasks show that task arithmetic is a simple, efficient and effective way of editing models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
