合成人格:LLM 如何使用社会经济微观数据模仿个体受访者?
Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?
AI总结:
研究利用德国社会经济面板数据构建个体级数字孪生,通过评估不同构建方法(模型、信息深度、嵌入方式、推理模式)对200万以上孪生响应的准确性,发现信息深度在75%熵分位数达到成本效益帕累托点,最佳单元准确率达78.8%。
摘要:
基于LLM的数字孪生有望扩展和加速市场研究,但大多数已发表的孪生要么是基于少数人口统计问题的粗略角色机器人,要么是基于专门收集的调查和访谈记录构建的详细个体级孪生。这两种设置都不涉及营销实践中操作上最相关的情况:从企业通过CRM系统、忠诚度计划和重复调查积累的现有异构面板数据中构建详细的个体孪生。我们从德国社会经济面板(SOEP)构建详细的个体级孪生,并在一个$3 \times 5 \times 2 \times 2$的构建方法网格中评估它们,该网格涵盖三个开放权重的LLM、五个按归一化香农熵排序的累积信息深度、两种嵌入方法和两种推理模式,对500名参与者和183个保留问题评分超过210万个孪生响应。孪生质量随信息深度提高,但超过75%熵分位数后收益递减,该分位数相对于性能最佳的100%单元充当成本效益帕累托点。将嵌入从叙述性角色摘要切换到原始对话历史(过去响应)在100%深度下每个模型-推理单元中提高了保留准确率,而显式思考模式提高了秩次相关性但不改变准确率。最佳单元准确率达到78.8%,Fisher-$z$相关性在SOEP保留评估集上达到$r = 0.590$。研究结果表明,基于孪生的市场研究不再受数据设计限制,而是受项目数量、模型选择和本文现在映射的一小部分构建级决策限制。
英文摘要:
LLM-based digital twins promise to scale and accelerate market research, but most published twins are either coarse persona bots conditioned on a few demographic questions or detailed individual-level twins built on purpose-collected surveys and interview transcripts. Neither setup speaks to the operationally most relevant case for marketing practice: building detailed individual twins from the pre-existing heterogeneous panel data that firms already accumulate through CRM systems, loyalty programs, and repeat surveys. We construct detailed individual-level twins from the German Socio-Economic Panel (SOEP) and evaluate them across a $3 \times 5 \times 2 \times 2$ construction-method grid that covers three open-weights LLMs, five cumulative information depths ranked by normalized Shannon entropy, two embedding methods, and two reasoning modes, scoring over 2.1 million twin responses on 500 participants and 183 held-out questions. Twin quality rises with information depth but with diminishing returns past the 75 percent entropy quartile, which acts as a cost-efficient Pareto point relative to the best-performing 100 percent cells. Switching the embedding from a narrative persona summary to a raw dialog history of past responses raises hold-out accuracy in every model-by-reasoning cell at the 100 percent depth, while an explicit thinking mode raises rank-order correlation without moving accuracy. Best-cell accuracy reaches 78.8 percent and Fisher-$z$ correlation reaches $r = 0.590$ on the SOEP held-out evaluation set. The findings suggest that twin-based market research is no longer gated by data design, but by item volume, model selection, and a small set of construction-level decisions that this paper now maps.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。