arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.05262cs.CLcs.LG
收藏

在GPT中查找和编辑事实关联

Locating and Editing Factual Associations in GPT

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Northeastern University(东北大学);Technion – IIT(以色列理工学院)

Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov

摘要:

我们分析了自回归Transformer语言模型中事实关联的存储和回忆,发现这些关联对应于本地化的,可直接编辑的计算的证据。我们首先开发一种因果干预,用于识别在模型的事实预测中起决定性作用的神经元激活。这揭示了中间层前馈模块中的一组独特的步骤,这些步骤在处理主题标记的同时介导事实预测。为了检验我们的假设,这些计算对应于事实关联回忆,我们修改前馈权重更新特定的事实关联使用排名一模型编辑(罗马)。我们发现,罗马是一个标准的zero-shot关系抽取(zsRE)模型编辑任务,有效的现有方法相比。为了进行更敏感的评估,我们还在一个新的反事实断言数据集上评估了ROME,在这个数据集上,它同时保持了特异性和泛化性,而其他方法则牺牲了一个或另一个。我们的研究结果证实了中间层前馈模块在存储事实关联中的重要作用,并表明直接操纵计算机制可能是模型编辑的一种可行方法。代码、数据集、可视化和交互式演示笔记本可在https://rome.baulab.info/上获得

英文摘要:

We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model's factual predictions. This reveals a distinct set of steps in middle-layer feed-forward modules that mediate factual predictions while processing subject tokens. To test our hypothesis that these computations correspond to factual association recall, we modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another. Our results confirm an important role for mid-layer feed-forward modules in storing factual associations and suggest that direct manipulation of computational mechanisms may be a feasible approach for model editing. The code, dataset, visualizations, and an interactive demo notebook are available at https://rome.baulab.info/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑