InCharacter：通过心理访谈评估角色扮演智能体的人格保真度
InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews
AI总结：
本文提出InCharacter方法，通过心理访谈和心理学量表评估角色扮演智能体的人格保真度，实验覆盖32个角色和14个量表，证明现有RPA人格与人类感知高度一致，准确率达80.7%。
摘要：
由大语言模型驱动的角色扮演智能体（RPA）已成为一个蓬勃发展的应用领域。然而，一个关键挑战在于评估RPA是否准确再现了目标角色的人物设定，即其角色保真度。现有方法主要关注角色的知识和语言模式。本文则引入了一种新的视角，利用心理学量表评估RPA的人格保真度。为克服以往对RPA进行自我报告评估的缺陷，我们提出了InCharacter，即对角色智能体进行访谈以进行人格测试。实验涵盖多种类型的RPA和大语言模型，涉及32个不同角色和14个广泛使用的心理学量表。结果验证了InCharacter在测量RPA人格方面的有效性。随后，借助InCharacter，我们表明最先进的RPA表现出与人类感知的角色人格高度一致，准确率最高可达80.7%。
英文摘要：
Role-playing agents (RPAs), powered by large language models, have emerged as a flourishing field of applications. However, a key challenge lies in assessing whether RPAs accurately reproduce the personas of target characters, namely their character fidelity. Existing methods mainly focus on the knowledge and linguistic patterns of characters. This paper, instead, introduces a novel perspective to evaluate the personality fidelity of RPAs with psychological scales. Overcoming drawbacks of previous self-report assessments on RPAs, we propose InCharacter, namely Interviewing Character agents for personality tests. Experiments include various types of RPAs and LLMs, covering 32 distinct characters on 14 widely used psychological scales. The results validate the effectiveness of InCharacter in measuring RPA personalities. Then, with InCharacter, we show that state-of-the-art RPAs exhibit personalities highly aligned with the human-perceived personalities of the characters, achieving an accuracy up to 80.7%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
