LLM模拟人格中的缩放定律:更详细和真实的人设档案即所需的一切
Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Need
AI总结:
本研究提出LLM虚拟人格评估框架,证明人设细节是关键,发现人格模拟中的缩放定律,为社会科学实验应用提供理论基础。
摘要:
本研究聚焦于利用大型语言模型(LLMs)模拟社会实验,探索其在虚拟人设角色扮演中模拟人类人格的能力。研究开发了一个端到端的评估框架,包括个体层面的稳定性和可识别性分析,以及称为渐进人格曲线的群体层面分析,以检验LLMs在模拟人类人格时的真实性和一致性。在方法论上,本研究对传统心理测量方法(CFA和构念效度)提出了重要修改,这些方法无法捕捉LLMs在当前低水平模拟中的改进趋势,可能导致过早拒绝或方法论错位。本研究的主要贡献包括:提出了LLM虚拟人格评估的系统框架;实证证明了人设细节在人格模拟质量中的关键作用;识别了人设档案的边际效用效应,特别是LLM人格模拟中的缩放定律,为在社会科学实验中应用大型语言模型提供了可操作的评估指标和理论基础。
英文摘要:
This research focuses on using large language models (LLMs) to simulate social experiments, exploring their ability to emulate human personality in virtual persona role-playing. The research develops an end-to-end evaluation framework, including individual-level analysis of stability and identifiability, as well as population-level analysis called progressive personality curves to examine the veracity and consistency of LLMs in simulating human personality. Methodologically, this research proposes important modifications to traditional psychometric approaches (CFA and construct validity) which are unable to capture improvement trends in LLMs at their current low-level simulation, potentially leading to remature rejection or methodological misalignment. The main contributions of this research are: proposing a systematic framework for LLM virtual personality evaluation; empirically demonstrating the critical role of persona detail in personality simulation quality; and identifying marginal utility effects of persona profiles, especially a Scaling Law in LLM personality simulation, offering operational evaluation metrics and a theoretical foundation for applying large language models in social science experiments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。