arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.06071cs.CLcs.AIcs.HC

你生命中的故事:基于丰富心理测量资料的LLM生成生命故事的往返评估

Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profiles

LoveMind AI;Jheronimus Academy of Data Science(耶罗尼米斯数据科学学院)

Ben Wigler, Maria Tsfasman, Tiffany Matej Hrkalovic

AI总结:

本文通过LLM生成生命故事并回收心理测量数据,验证了LLM在编码和解码个体差异方面的鲁棒性,展示了心理特征与语言表达之间的关系。

摘要:

人格特质在自然语言中丰富地编码,大型语言模型(LLM)在基于人物描述训练后可以模拟人格。然而,现有评估主要依赖于条件模型的问卷自报,建筑多样性有限,很少使用真实人类心理测量数据。本文通过将LLM条件于290名参与者的丰富心理测量资料生成第一人称生命故事叙述,然后让独立的LLM从这些叙述中恢复人格分数。结果显示,生成的叙述中的人格分数恢复水平接近人类测试-再测可靠性(均值r=0.750,85%的人类天花板),且在10个LLM叙述生成器和3个LLM人格评分器中表现稳健。分解系统偏差揭示,评分模型在抵消对齐引起的默认设置的同时实现准确性。生成叙述的内容分析显示,人格条件生成的行为差异化文本:十个编码特征中的九个与参与者的真实对话中的相同特征显著相关,且人格驱动的情感反应模式在真实对话数据中复制。这些发现提供了证据,表明预训练期间捕捉到的人格-语言关系支持了个体差异的稳健编码和解码,包括在真实人类行为中复制的特征性情绪变化模式。

英文摘要:

Personality traits are richly encoded in natural language, and large language models (LLMs) trained on human text can simulate personality when conditioned on persona descriptions. However, existing evaluations rely predominantly on questionnaire self-report by the conditioned model, are limited in architectural diversity, and rarely use real human psychometric data. Without addressing these limitations, it remains unclear whether personality conditioning produces psychometrically informative representations of individual differences or merely superficial alignment with trait descriptors. To test how robustly LLMs can encode personality into extended text, we condition LLMs on real psychometric profiles from 290 participants to generate first-person life story narratives, and then task independent LLMs to recover personality scores from those narratives alone. We show that personality scores can be recovered from the generated narratives at levels approaching human test-retest reliability (mean r = 0.750, 85% of the human ceiling), and that recovery is robust across 10 LLM narrative generators and 3 LLM personality scorers spanning 6 providers. Decomposing systematic biases reveals that scoring models achieve their accuracy while counteracting alignment-induced defaults. Content analysis of the generated narratives shows that personality conditioning produces behaviourally differentiated text: nine of ten coded features correlate significantly with the same features in participants' real conversations, and personality-driven emotional reactivity patterns in narratives replicate in real conversational data. These findings provide evidence that the personality-language relationship captured during pretraining supports robust encoding and decoding of individual differences, including characteristic emotional variability patterns that replicate in real human behaviour.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑