发表机构
IT:U, Interdisciplinary Transformation University Austria（奥地利跨学科转型大学（IT:U））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究测试六个LLMs模拟人类信念更新的能力，发现Qwen3-32B和GPT-5-Mini仅在提供人类实际初始立场时能匹配人类后立场分布，所有模型存在系统性偏差，模拟仅在现实起始条件下可靠。
AI 中文摘要
大型语言模型（LLMs）越来越多地被用作社会科学实验中人类研究参与者的替代者，但这种做法的保真度很少被直接测试。我们测试了六个LLMs是否可以模拟个体人类的信念更新，将LLM的输出与来自Prolific平台上391名英国参与者的真实数据进行一对一比较，这些参与者在阅读Reddit评论后更新了他们对三个讨论话题的立场。每个参与者都由一个LLM模拟，该LLM的条件是基于其人口统计和人格特质数据生成的角色设定。我们发现，一些LLMs（Qwen3-32B和GPT-5-Mini）可以匹配人类的后立场分布，但只有在提供参与者实际初始立场时才能做到。所有六个模型都无法自行模拟初始立场，也无法从自我生成的立场中产生忠实的信念更新。所有模型都出现了三种系统性偏差：中立立场的过度代表、比人类更频繁但幅度更小的信念转变，以及无法按说服力对评论进行排名。人口统计和人格特质角色设定对保真度没有一致影响。LLM对人类信念动态的模拟只有在基于现实起始条件时才可靠，而当前的多轮社交媒体模拟很少提供这种条件。
英文摘要
LLMs are increasingly deployed as proxies for human study participants in social science experiments, yet the fidelity of this practice has rarely been tested directly. We test whether six LLMs can simulate individual human belief updates, comparing LLM outputs 1-to-1 against ground truth data from 391 UK participants on Prolific, who updated their stances on three discussion topics after reading Reddit comments. Each participant was simulated by an LLM conditioned on a persona derived from their demographic and personality trait data. We find that some LLMs (Qwen3-32B and GPT-5-Mini) can match the human post-stance distribution, but only when given participants' actual initial stances. All six models fail to simulate initial stances themselves and to produce faithful belief updates from self-generated stances. Three systematic biases emerge across all models: overrepresentation of neutral positions, more frequent but smaller belief shifts than humans, and a failure to rank comments by convincingness. Demographic and personality trait personas had no consistent effect on fidelity. LLM simulations of human belief dynamics are only reliable when grounded in realistic starting conditions, that current multi-round social media simulations rarely provide.
