arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00016cs.CLcs.AI

PTCBENCH: 对LLM系统中人格特质情境稳定性的基准测试

PTCBENCH: Benchmarking Contextual Stability of Personality Traits in LLM Systems

Singapore Management University(新加坡管理大学);Tianjin University(天津大学);Nanyang Technological University(南洋理工大学);Xi’an Jiaotong University(西安交通大学)

Jiongchi Yu, Yuhan Ma, Xiaoyu Zhang, Junjie Wang, Qiang Hu, Chao Shen, Xiaofei Xie

AI总结:

PTCBENCH通过评估LLM在不同情境下的人格一致性,揭示外部事件对LLM人格和推理能力的影响,为构建更稳健的心理学对齐AI系统提供新视角。

摘要:

随着大型语言模型(LLMs)在情感代理和人工智能系统中的广泛应用,保持一致且真实的LLM人格变得对用户信任和参与至关重要。然而,现有研究忽视了一个人格特质是动态且情境依赖的的基本心理共识。为弥补这一差距,我们引入了PTCBENCH,一个系统化的基准测试,用于量化在受控情境下LLM人格的一致性。PTCBENCH让模型经历12种不同的外部条件,涵盖多样的地理位置和生活事件,并严格使用NEO五因素量表评估人格。我们的研究基于39,240条人格特质记录揭示,某些外部情境(如“失业”)可以触发LLM显著的人格变化,甚至改变其推理能力。总体而言,PTCBENCH建立了一个可扩展的框架,用于在现实和演变的环境中评估人格一致性,为开发稳健且心理契合的AI系统提供可操作的见解。

英文摘要:

With the increasing deployment of large language models (LLMs) in affective agents and AI systems, maintaining a consistent and authentic LLM personality becomes critical for user trust and engagement. However, existing work overlooks a fundamental psychological consensus that personality traits are dynamic and context-dependent. To bridge this gap, we introduce PTCBENCH, a systematic benchmark designed to quantify the consistency of LLM personalities under controlled situational contexts. PTCBENCH subjects models to 12 distinct external conditions spanning diverse location contexts and life events, and rigorously assesses the personality using the NEO Five-Factor Inventory. Our study on 39,240 personality trait records reveals that certain external scenarios (e.g., "Unemployment") can trigger significant personality changes of LLMs, and even alter their reasoning capabilities. Overall, PTCBENCH establishes an extensible framework for evaluating personality consistency in realistic, evolving environments, offering actionable insights for developing robust and psychologically aligned AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑