预训练Transformer中的知识回路
Knowledge Circuits in Pretrained Transformers
AI总结：
本文通过分析GPT2和TinyLLAMA的计算图，揭示由信息头、关系头和多层感知机协同构成的知识回路，并利用其评估知识编辑方法及解释幻觉和上下文学习行为。
摘要：
现代大语言模型的卓越能力植根于其参数中编码的海量知识库，使其能够感知世界并进行推理。这些模型如何存储知识的内部工作机制长期以来一直是研究者高度关注和探究的课题。迄今为止，大多数研究集中于这些模型中的孤立组件，例如多层感知机和注意力头。在本文中，我们深入语言模型的计算图，以揭示在表达特定知识中起关键作用的知识回路。在GPT2和TinyLLAMA上进行的实验使我们能够观察到某些信息头、关系头和多层感知机如何在模型内协同编码知识。此外，我们评估了当前知识编辑技术对这些知识回路的影响，为这些编辑方法的功能和局限性提供了更深入的见解。最后，我们利用知识回路来分析和解释语言模型行为，如幻觉和上下文学习。我们相信知识回路在推进我们对Transformer的理解以及指导知识编辑的改进设计方面具有潜力。代码和数据可在https://github.com/zjunlp/KnowledgeCircuits获取。
英文摘要：
The remarkable capabilities of modern large language models are rooted in their vast repositories of knowledge encoded within their parameters, enabling them to perceive the world and engage in reasoning. The inner workings of how these models store knowledge have long been a subject of intense interest and investigation among researchers. To date, most studies have concentrated on isolated components within these models, such as the Multilayer Perceptrons and attention head. In this paper, we delve into the computation graph of the language model to uncover the knowledge circuits that are instrumental in articulating specific knowledge. The experiments, conducted with GPT2 and TinyLLAMA, have allowed us to observe how certain information heads, relation heads, and Multilayer Perceptrons collaboratively encode knowledge within the model. Moreover, we evaluate the impact of current knowledge editing techniques on these knowledge circuits, providing deeper insights into the functioning and constraints of these editing methodologies. Finally, we utilize knowledge circuits to analyze and interpret language model behaviors such as hallucinations and in-context learning. We believe the knowledge circuits hold potential for advancing our understanding of Transformers and guiding the improved design of knowledge editing. Code and data are available in https://github.com/zjunlp/KnowledgeCircuits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
