超越对话：面向通用角色扮演语言模型的角色设定-对话对齐框架
BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
AI总结：
针对角色扮演训练中对话与角色设定冲突及句子级对齐缺失问题，本文提出 BEYOND DIALOGUE 自动化框架，通过超越对话任务和推理提示实现细粒度对齐，并在实验中超越多数通用与专用基线。
摘要：
大语言模型（LLMs）的快速发展彻底革新了角色扮演，推动了通用角色扮演模型的开发。然而，当前的角色扮演训练存在两个显著问题：（I）使用预定义角色设定来提示特定场景的对话训练，通常会导致对话与角色设定之间不一致甚至冲突，从而产生训练偏差。（II）模型仅学习基于角色设定模仿角色，忽视了句子层面的角色设定-对话对齐。在本研究中，我们提出了一个简单而有效的框架，名为 BEYOND DIALOGUE，旨在克服这些障碍。该框架创新性地引入“超越对话”任务，根据每个特定场景使对话与角色特征对齐，从而消除训练过程中的偏差。此外，通过采用一种创新的提示机制来生成用于训练的推理结果，该框架使模型能够在句子层面实现角色设定与对话之间的细粒度对齐。上述方法完全自动化且成本低廉。此外，自动化对话与客观评估方法的整合形成了一个综合框架，为通用角色扮演铺平了道路。实验结果表明，我们的模型在遵循和体现角色设定各个维度方面表现出色，优于大多数专有通用和专用角色扮演基线模型。所有代码和数据集均可在 https://github.com/yuyouyu32/BeyondDialogue 获取。
英文摘要：
The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. In this work, we propose a simple yet effective framework called BEYOND DIALOGUE, designed to overcome these hurdles. This framework innovatively introduces "beyond dialogue" tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level. The aforementioned methods are fully automated and low-cost. Additionally, the integration of automated dialogue and objective evaluation methods forms a comprehensive framework, paving the way for general role-playing. Experimental results demonstrate that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines. All code and datasets are available at https://github.com/yuyouyu32/BeyondDialogue.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
