记忆即一切：检验模型记忆如何影响大语言模型在标注任务中的表现
Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Tasks
AI总结：
本文通过GPT-4o和Llama 3.1在两个政治科学数据集上的实验，证明模型记忆能显著提升LLM标注性能，并提出结合记忆与强化学习的记忆强化方法，进一步改善多数测试结果。
摘要：
生成式大语言模型（LLMs）在使用零样本和少样本学习进行文本标注时已展现出有前景的结果。然而，这些方法不允许模型保留先前标注中的信息，使得每次响应都独立于此前的响应。这引出了一个问题：模型记忆——即大语言模型对自身在同一任务中先前标注的了解——是否会影响性能。在本文中，我们使用OpenAI的GPT-4o和Meta的Llama 3.1，在两个政治科学数据集上进行实验，证明允许模型保留关于自身先前分类的信息能带来显著的性能提升：与零样本和少样本学习相比，提升幅度在5%到25%之间。此外，我们提出了一种新方法——记忆强化（memory reinforcement），它结合了模型记忆与强化学习，在我们的四项测试中有三项带来了额外的性能提升。这些发现对希望提高大语言模型标注任务性能和效率的应用研究者具有重要意义。
英文摘要：
Generative Large Language Models (LLMs) have shown promising results in text annotation using zero-shot and few-shot learning. Yet these approaches do not allow the model to retain information from previous annotations, making each response independent from the preceding ones. This raises the question of whether model memory -- the LLM having knowledge about its own previous annotations in the same task -- affects performance. In this article, using OpenAI's GPT-4o and Meta's Llama 3.1 on two political science datasets, we demonstrate that allowing the model to retain information about its own previous classifications yields significant performance improvements: between 5 and 25\% when compared to zero-shot and few-shot learning. Moreover, memory reinforcement, a novel approach we propose that combines model memory and reinforcement learning, yields additional performance gains in three out of our four tests. These findings have important implications for applied researchers looking to improve performance and efficiency in LLM annotation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
