UniMS-RAG：面向个性化对话系统的统一多源检索增强生成
UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems
AI总结：
针对个性化对话系统的多源信息整合问题，提出UniMS-RAG系统，将知识源选择、检索和响应生成统一到序列范式，利用特殊标记交互多源并评估相关性，结合自优化机制，在DuLeMon和KBP数据集上取得SOTA性能。
摘要：
大型语言模型（LLMs）在许多自然语言理解和生成任务中展现出卓越能力。然而，个性化问题仍是一个备受追捧的特性，尤其是在对话系统涉及多源信息时。为了更好地规划并整合多源信息以生成个性化响应，我们首先将其分解为三个子任务：知识源选择、知识检索和响应生成。随后，我们提出一种新颖的统一多源检索增强生成系统（UniMS-RAG）。具体而言，我们在训练过程中通过不同的公式化方式将这三个子任务统一到相同的序列到序列范式中，利用名为动作标记和评估标记的特殊标记自适应地检索证据并按需评估相关性。让语言模型生成动作标记有助于与各种知识源交互，使其行为适应不同任务需求；同时，评估标记衡量对话上下文与检索到的证据之间的相关度得分。此外，我们精心设计了一种自优化机制，考虑以下两点来迭代优化生成的响应：1）生成的响应与检索到的证据之间的一致性得分；2）相关度得分。在两个个性化数据集（DuLeMon和KBP）上的实验表明，UniMS-RAG在知识源选择和响应生成任务中以自身作为检索器的统一方式实现了最先进的性能。我们还提供了广泛的分析和讨论，为个性化对话系统提供了一些新视角。
英文摘要：
Large Language Models (LLMs) has shown exceptional capabilities in many natual language understanding and generation tasks. However, the personalization issue still remains a much-coveted property, especially when it comes to the multiple sources involved in the dialogue system. To better plan and incorporate the use of multiple sources in generating personalized response, we firstly decompose it into three sub-tasks: Knowledge Source Selection, Knowledge Retrieval, and Response Generation. We then propose a novel Unified Multi-Source Retrieval-Augmented Generation system (UniMS-RAG) Specifically, we unify these three sub-tasks with different formulations into the same sequence-to-sequence paradigm during the training, to adaptively retrieve evidences and evaluate the relevance on-demand using special tokens, called acting tokens and evaluation tokens. Enabling language models to generate acting tokens facilitates interaction with various knowledge sources, allowing them to adapt their behavior to diverse task requirements. Meanwhile, evaluation tokens gauge the relevance score between the dialogue context and the retrieved evidence. In addition, we carefully design a self-refinement mechanism to iteratively refine the generated response considering 1) the consistency scores between the generated response and retrieved evidence; and 2) the relevance scores. Experiments on two personalized datasets (DuLeMon and KBP) show that UniMS-RAG achieves state-of-the-art performance on the knowledge source selection and response generation task with itself as a retriever in a unified manner. Extensive analyses and discussions are provided for shedding some new perspectives for personalized dialogue systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
