评估大语言模型（LLMs）模拟真实人类人格的效能
Evaluating the Efficacy of LLMs to Emulate Realistic Human Personalities
AI总结：
本研究以超5万份人类人格调查数据为基准，对比前沿与本地LLMs，发现部分前沿模型可100%对齐人类人格，为游戏开发者提供了测试LLMs模拟人类人格的方法。
摘要：
为提升电子游戏中的沉浸感与参与度，情感型非玩家角色（Affective Non-Player Characters，ANPCs）的设计是研究者与从业者的核心关注点。情感计算框架通过赋予非玩家角色（Non-player characters，NPC）人格、情感与社会关系来优化其表现。大语言模型（Large Language Models，LLMs）与这些框架结合时，有望动态优化角色设计，但需进一步研究以验证这些模型是否真正体现人类特质。本研究开展全面分析，利用大五人格模型（Big Five）与来自国际人格项目池（International Personality Item Pool，IPIP）问卷的人类反应，探究LLMs生成与人类人格相符内容的能力。我们的目标是将各类LLMs（包括前沿模型与本地模型）的性能，与包含超5万份自我报告人格测试人类调查的大规模数据集进行基准对比，以确定LLMs能否在人格驱动提示下复制类人决策。我们采用一系列人格画像对人类调查数据集的测试结果进行聚类，方法是针对每个人格画像，用自我评估测试项提示LLMs，将其输出与人类基线反应对比，并评估准确性与一致性。研究发现，部分本地模型与人类数据集的任何人格画像对齐度为0%，而前沿模型在部分情况下对齐度达100%。结果表明，通过将LLMs输出与人类数据基准对比，NPC可借助LLMs成功模拟类人人格特质。这项基础工作为游戏开发者与研究者提供了测试和评估LLMs的方法，确保其准确呈现所需人类人格，并可扩展用于进一步验证。
英文摘要：
To enhance immersion and engagement in video games, the design of Affective Non-Player Characters (ANPCs) is a key focus for researchers and practitioners. Affective Computing frameworks improve Non-player characters (NPC) by providing personalities, emotions, and social relations. Large Language Models (LLMs) bring the promise to dynamically enhance character design when coupled with these frameworks, but further research is needed to validate the models truly represent human qualities. In this research, a comprehensive analysis investigates the capabilities of LLMs to generate content that aligns with human personality, using the Big Five and human responses from the International Personality Item Pool (IPIP) questionnaire. Our goal is to benchmark the performance of various LLMs, including frontier models and local models, against an extensive dataset comprising over 50,000 human surveys of self-reported personality tests to determine whether LLMs can replicate human-like decision-making with personality-driven prompts. A range of personality profiles were used to cluster the test results from the human survey dataset. Our methodology involved prompting LLMs with self-evaluated test items for each personality profile, comparing their outputs to human baseline responses, and evaluating the accuracy and consistency. Our findings show that some local models had 0% alignment of any personality profiles when compared to the human dataset, while the frontier models, in some cases, had 100% alignment. The results indicate that NPCs can successfully emulate human-like personality traits using LLMs, as demonstrated by benchmarking the LLM's output against human data. This foundational work serves as a methodology for game developers and researchers to test and evaluate LLMs, ensuring they accurately represent the desired human personalities and can be expanded for further validation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
