重新思考角色扮演评估：匿名基准测试与对性格效应的系统研究
Rethinking Role-Playing Evaluation: Anonymous Benchmarking and a Systematic Study of Personality Effects
AI总结：
本文提出匿名基准测试方法，揭示角色扮演能力与角色识别的关联，通过性格增强方法提升匿名场景下的角色扮演性能。
摘要：
大型语言模型（LLMs）在开发角色扮演代理（RPAs）方面展现出显著潜力。然而，当前评估框架依赖于知名虚构角色，引发关键担忧：模型可能依赖内部训练记忆而非展示角色扮演能力。这种依赖常导致当RPAs遇到未见过或分布外的个性时性能下降。为此，我们提出更严谨的评估协议，旨在分离角色扮演熟练度与角色识别。我们的实验在多个基准上显示，匿名化角色会降低性能，证实名称暴露提供隐含线索，掩盖模型的真实能力。为缓解此问题，我们研究多样化的个性增强作为提升匿名场景下角色忠实度的方法。我们系统分析了各种个性描述方法对代理行为和一致性的影响。结果表明，整合个性信息能一致提升RPA性能。本文建立了更公平的评估标准，并验证了可扩展的、基于个性的框架，用于构建稳健的RPAs。
英文摘要：
Large Language Models (LLMs) have shown remarkable potential in developing role-playing agents (RPAs). However, current evaluation frameworks rely heavily on well-known fictional characters, raising a critical concern: models may be leveraging their internal training memory of these characters rather than demonstrating role-playing capabilities. This reliance often leads to significant performance degradation when RPAs encounter unseen or out-of-distribution personas. To address this, we propose a more rigorous evaluation protocol designed to decouple role-playing proficiency from character recognition. Our experiments across multiple benchmarks demonstrate that anonymizing characters degrades performance, confirming that name exposure provides implicit cues that mask a model's true capability. To mitigate this, we investigate diverse personality augmentation as a method to enhance role fidelity in anonymous settings. We systematically analyze the impact of various personality-description methods on agent behavior and consistency. Our results show that incorporating personality information consistently improves RPA performance. This work establishes a more equitable evaluation standard and validates a scalable, personality-enhanced framework for constructing robust RPAs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
