发表机构
Istanbul Technical University(伊斯坦布尔技术大学)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
研究非怪异LLM角色建模中独立代理范式的问题,提出分布优先校正方法,通过言语化采样等进行实验,发现原范式有崩溃等问题,校正方法能测量内部不一致性及校准条件。
AI 中文摘要
合成总体工具越来越多地将每个个体作为独立的大语言模型(LLM)代理运行。利用实际调查微观数据,我们表明这种范式存在基本失败模式,并针对其设置了分布优先校正,所有这些都使用确定性、经过结构验证的验证器在非怪异(以土耳其为先)数据上进行测量。首先,以2414名真实世界价值观调查受访者为基础的N个独立LLM代理无法再现总体的响应分布。其次,言语化采样(VS)在三个模型家族中无需训练即可修复该领域长期存在的欠分散问题,但同样会普遍过度分散。第三,调查保真度仅微弱地转移到代理行为上。第四,安慰剂对照记忆攻击和选举回测表明VS保持总体强度,而子群体和个体的主张受到召回和不确定性的影响。最后我们提出校正方法:一次性对分布进行建模(VS)并以O(1)成本将其分配给有基础的角色,使用预算感知路由器,其真实AUC为0.805。中心贡献无需现实主义主张:它测量了独立代理路线的内部不一致性以及分布优先路线进行校准的条件。
英文摘要
Synthetic-population tools increasingly run every individual as an independent large language model (LLM) agent. Using real survey microdata, we show that this paradigm has a basic failure mode, and we set a distribution-first corrective against it, all measured with a deterministic, construct-validated verifier on non-WEIRD (Turkey-first) data. First, N independent LLM agents grounded on 2,414 real World Values Survey respondents fail to reproduce the population's response distribution: they pile onto a modal default (four scenarios x five seeds: concentration 0.36->0.69, entropy 1.46->0.77, 85% collapse, TVD=0.44), and the collapse is a predictable function of scenario structure (r=0.55 with a single-answer structure). Second, Verbalized Sampling (VS) fixes the field's chronic under-dispersion without training in three model families (fidelity +7 to +10; significant on Qwen, p=0.002, d=6.2), yet the same move universally overshoots into over-dispersion (SD-ratio 0.4-0.56 -> 1.26-1.37), a structural property of VS. Third, survey fidelity transfers only weakly to agentic behavior: in a single-model, single-domain booking task, a persona is dominated by a cheapest-default (~80%) that income modulates but does not override (comfort choice 0%->7%->32% across income bands). Fourth, a placebo-controlled memorization attack and an election backtest show VS keeps aggregate strength while subgroup and individual claims are contaminated by recall and underdetermination. We close with the corrective: model the distribution once (VS) and assign it to grounded characters at O(1) cost, with a budget-aware router whose honest AUC is 0.805, not the tautological 1.0 of a code-derived oracle. The central contribution needs no realism claim: it measures the internal inconsistency of the independent-agent route and the conditions under which the distribution-first route calibrates.
Comments8 pages, 8 figures