上下文示例作为从大型联想记忆中检索的线索
In-Context Exemplars as Clues to Retrieving from Large Associative Memory
AI总结：
该研究将大语言模型的上下文学习（ICL）概念化为联想记忆的上下文检索，基于Hopfield Networks构建理论框架，探究示例对ICL性能的影响并提出高效主动示例选择方法，为理解ICL机制及大语言模型提供新视角。
摘要：
近年来，大语言模型（LLM）在自然语言处理领域取得了显著进展。大语言模型最具代表性的能力是上下文学习（ICL），它使大语言模型无需训练即可从上下文中的示例里学习模式。上下文学习的性能在很大程度上取决于所使用的示例。然而，由于对上下文学习的工作机制缺乏了解，如何选择示例仍不明确。本文提出了一种关于上下文学习的新视角，将其概念化为从联想记忆模型中进行的上下文检索。我们基于Hopfield Networks（霍普菲尔德网络）建立了上下文学习的理论框架。基于该框架，我们研究了上下文示例如何影响上下文学习的性能，并提出了更高效的主动示例选择方法。本研究通过将上下文学习与记忆检索联系起来，为其机制提供了新的见解，对推进大语言模型的理解具有潜在意义。
英文摘要：
Recently, large language models (LLMs) have made remarkable progress in natural language processing. The most representative ability of LLMs is in-context learning (ICL), which enables LLMs to learn patterns from in-context exemplars without training. The performance of ICL greatly depends on the exemplars used. However, how to choose exemplars remains unclear due to the lack of understanding of how in-context learning works. In this paper, we present a novel perspective on ICL by conceptualizing it as contextual retrieval from a model of associative memory. We establish a theoretical framework of ICL based on Hopfield Networks. Based on our framework, we look into how in-context exemplars influence the performance of ICL and propose more efficient active exemplar selection. Our study sheds new light on the mechanism of ICL by connecting it to memory retrieval, with potential implications for advancing the understanding of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
