论使用大型语言模型进行角色扮演时的决策能力
On the Decision-Making Abilities in Role-Playing using Large Language Models
AI总结：
本文构建16种MBTI虚拟角色并从适应性、探索-利用权衡、推理和安全四方面量化评估LLM角色扮演后的决策能力，发现其决策表现与角色类型稳定相关。
摘要：
大型语言模型（LLM）如今越来越多地用于角色扮演任务，尤其是通过角色扮演提示来扮演特定领域专家。在真实场景中交互时，角色的决策能力会显著塑造其行为模式。本文聚焦于评估LLM在角色扮演后的决策能力，从而验证角色扮演的有效性。我们的目标是为提升LLM在角色扮演任务中的决策能力提供指标和指导。具体而言，我们首先使用LLM生成与Myers-Briggs Type Indicator（缩写为MBTI）的16种人格类型相对应的虚拟角色描述，这些类型代表对人群的一种划分。随后，我们设计具体的量化操作，从适应性、探索与利用权衡能力、推理能力和安全性四个方面评估LLM在角色扮演后的决策能力。最后，我们通过GPT-4分析决策表现与相应MBTI类型之间的关联。大量实验表明，不同角色在决策能力的四个方面存在稳定差异，表明决策能力与LLM所模拟角色之间具有稳健相关性。这些结果强调，LLM能够有效扮演不同角色，同时体现其真实的社会学特征。
英文摘要：
Large language models (LLMs) are now increasingly utilized for role-playing tasks, especially in impersonating domain-specific experts, primarily through role-playing prompts. When interacting in real-world scenarios, the decision-making abilities of a role significantly shape its behavioral patterns. In this paper, we concentrate on evaluating the decision-making abilities of LLMs post role-playing thereby validating the efficacy of role-playing. Our goal is to provide metrics and guidance for enhancing the decision-making abilities of LLMs in role-playing tasks. Specifically, we first use LLMs to generate virtual role descriptions corresponding to the 16 personality types of Myers-Briggs Type Indicator (abbreviated as MBTI) representing a segmentation of the population. Then we design specific quantitative operations to evaluate the decision-making abilities of LLMs post role-playing from four aspects: adaptability, exploration$\&$exploitation trade-off ability, reasoning ability, and safety. Finally, we analyze the association between the performance of decision-making and the corresponding MBTI types through GPT-4. Extensive experiments demonstrate stable differences in the four aspects of decision-making abilities across distinct roles, signifying a robust correlation between decision-making abilities and the roles emulated by LLMs. These results underscore that LLMs can effectively impersonate varied roles while embodying their genuine sociological characteristics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
