提升自我：具有自记忆的检索增强文本生成
Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory
AI总结：
提出selfmem框架，利用自记忆迭代增强检索增强生成，在神经机器翻译、摘要和对话任务上取得最先进结果。
摘要：
通过直接访问人类撰写的参考作为记忆，检索增强生成在广泛的文本生成任务中取得了显著进展。由于更好的记忆通常会促进更好的生成（我们将其定义为原始问题），传统的记忆检索方法涉及选择与输入具有最高相似度的记忆。然而，这种方法受到检索记忆的固定语料库质量的限制。在本文中，通过探索原始问题的对偶性：更好的生成也促进更好的记忆，我们提出了一种新颖的框架，即selfmem，该框架通过迭代地使用检索增强生成器来创建无界记忆池，并使用记忆选择器选择一个输出作为后续生成轮次的记忆，从而解决了这一限制。这使得模型能够利用其自身的输出（称为自记忆）来改进生成。我们在三个不同的文本生成任务上评估了selfmem的有效性：神经机器翻译、抽象式文本摘要和对话生成，在两种生成范式下：微调的小模型和少样本LLM。我们的方法在JRC-Acquis、XSum（50.3 ROUGE-1）和BigPatent（62.9 ROUGE-1）的四个方向上取得了最先进的结果，展示了自记忆在增强检索增强生成模型方面的潜力。此外，我们对selfmem框架中的每个组件进行了彻底的分析，以识别瓶颈并为未来研究提供见解。
英文摘要：
With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation~(we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, selfmem, which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis, XSum (50.3 ROUGE-1), and BigPatent (62.9 ROUGE-1), demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the selfmem framework to identify bottlenecks and provide insights for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
