上下文学习的图谱：注意力头如何塑造上下文检索增强
The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation
AI总结：
本文通过归因方法识别上下文学习中的专门注意力头，揭示其理解指令、检索信息及存储知识的作用，并利用这些见解追踪知识来源，以促进更安全透明的语言模型。
摘要：
大型语言模型能够利用上下文学习，通过检索增强来获取训练数据之外的外部知识。尽管前景广阔，但其内部工作机制仍不清晰。在本工作中，我们通过将提示视为信息组件的组合，阐明了问答任务中上下文检索增强的机制。我们提出了一种基于归因的方法来识别专门的注意力头，揭示了理解指令并检索相关上下文信息的上下文头，以及存储实体关系知识的参数头。为了更好地理解它们的作用，我们提取函数向量并修改其注意力权重，以展示它们如何影响答案生成过程。最后，我们利用获得的见解来追踪推理过程中使用的知识来源，为构建更安全、更透明的语言模型铺平道路。
英文摘要：
Large language models are able to exploit in-context learning to access external knowledge beyond their training data through retrieval-augmentation. While promising, its inner workings remain unclear. In this work, we shed light on the mechanism of in-context retrieval augmentation for question answering by viewing a prompt as a composition of informational components. We propose an attribution-based method to identify specialized attention heads, revealing in-context heads that comprehend instructions and retrieve relevant contextual information, and parametric heads that store entities' relational knowledge. To better understand their roles, we extract function vectors and modify their attention weights to show how they can influence the answer generation process. Finally, we leverage the gained insights to trace the sources of knowledge used during inference, paving the way towards more safe and transparent language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
