大型语言模型是所有角色的叠加：通过自对齐实现任意角色扮演
Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
AI总结：
本文提出自对齐方法Ditto，利用LLM内在角色知识自动生成含4,000个角色的训练数据并微调模型，在多轮角色扮演中达到媲美专有聊天机器人的效果。
摘要：
为提升开源大型语言模型（LLM）的角色扮演能力，研究者已投入大量努力，通过模仿专有同类模型来实现这一目标。然而，我们认为LLM本身就具备角色扮演能力，因为其庞大训练语料中蕴含着关于角色和潜在对话的广泛知识。因此，本研究提出Ditto，一种用于角色扮演的自对齐方法。Ditto利用角色知识，促使遵循指令的LLM将角色扮演对话模拟为阅读理解的一种变体。该方法构建了一个包含4,000个角色的角色扮演训练集，在角色数量上比现有数据集大十倍。随后，我们使用这一自生成数据集微调LLM，以增强其角色扮演能力。在我们精心构建且可复现的角色扮演基准以及MT-Bench的角色扮演子集上评估时，Ditto在多种参数规模下均能在多轮角色扮演对话中持续保持一致的角色身份，并提供准确的角色专属知识。值得注意的是，它优于所有开源角色扮演基线，表现出与先进专有聊天机器人相当的性能。此外，我们呈现了角色扮演领域首个全面的跨监督对齐实验，揭示LLM的内在能力限制了角色扮演中的知识；同时，角色扮演风格可在较小模型的指导下轻松习得。我们在https://github.com/OFA-Sys/Ditto开源相关资源。
英文摘要：
Considerable efforts have been invested in augmenting the role-playing proficiency of open-source large language models (LLMs) by emulating proprietary counterparts. Nevertheless, we posit that LLMs inherently harbor role-play capabilities, owing to the extensive knowledge of characters and potential dialogues ingrained in their vast training corpora. Thus, in this study, we introduce Ditto, a self-alignment method for role-play. Ditto capitalizes on character knowledge, encouraging an instruction-following LLM to simulate role-play dialogues as a variant of reading comprehension. This method creates a role-play training set comprising 4,000 characters, surpassing the scale of currently available datasets by tenfold regarding the number of roles. Subsequently, we fine-tune the LLM using this self-generated dataset to augment its role-playing capabilities. Upon evaluating our meticulously constructed and reproducible role-play benchmark and the roleplay subset of MT-Bench, Ditto, in various parameter scales, consistently maintains a consistent role identity and provides accurate role-specific knowledge in multi-turn role-play conversations. Notably, it outperforms all open-source role-play baselines, showcasing performance levels comparable to advanced proprietary chatbots. Furthermore, we present the first comprehensive cross-supervision alignment experiment in the role-play domain, revealing that the intrinsic capabilities of LLMs confine the knowledge within role-play. Meanwhile, the role-play styles can be easily acquired with the guidance of smaller models. We open-source related resources at https://github.com/OFA-Sys/Ditto.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
