增强大语言模型角色扮演中人物一致性的基于人物感知的对比学习
Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning
AI总结：
提出人物感知对比学习（PCL）框架，通过角色链和迭代对比学习增强LLM角色扮演中的人物一致性，实验显示在自动和人工评估中均优于基线。
摘要：
近年来，大语言模型（LLMs）在许多对话生成任务中取得了突破性进展。然而，它们缺乏情感和细粒度角色意识，这进一步限制了模型提供个性化和多样化交互的能力。当前方法在角色扮演等场景中收集高质量标注数据的成本高昂，而传统的人类对齐方法由于角色扮演场景中模型行为的固有多样性而难以部署。受通过RLHF（基于人类反馈的强化学习）对模型进行安全行为对齐的启发，本文从人物对齐的角度重新审视模型角色扮演行为，并提出了一种新颖的无需标注的框架，名为人物感知对比学习（PCL），以对齐LLMs在角色扮演中的行为，增强模型的角色一致性。具体来说，我们首先设计了一种角色链方法，鼓励模型基于角色特征和对话上下文进行自我提问，以调整人格一致性。然后，我们通过在使用角色特征与不使用角色特征之间进行迭代对比学习，进一步增强模型的角色扮演策略。在黑盒和白盒LLMs上的实验表明，配备PCL的LLMs在自动评估方法（CharEval和GPT-4）以及人类专家评估下显著优于普通LLMs。
英文摘要：
In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model's ability to provide personalized and diverse interactions further. Current methods face high costs in collecting high-quality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios. Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named \textbf{\underline{P}}ersona-Aware \textbf{\underline{C}}ontrastive \textbf{\underline{L}}earning (PCL) to align LLMs' behavior during role-playing, enhancing the model's role consistency. Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency. Then, we further enhance the model's role-playing strategy through iterative contrastive learning between the use of role characteristics and not. Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval \& GPT-4) and human expert evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
