Vox Populi, Vox AI？使用语言模型估计德国公众意见
Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion
AI总结：
本研究以德国投票选择为例，使用GPT-3.5生成合成样本估计公众意见，发现其预测不准确且存在党派偏见，揭示了LLM在意见表征上的局限。
摘要：
大型语言模型（LLM）的最新发展引发了关于LLM生成的“合成样本”能否补充或替代传统调查的讨论，因为它们的训练数据可能反映了人群中普遍存在的态度和行为。许多主要基于美国的研究促使LLM模拟调查受访者，其中一些研究发现其回答与调查数据高度匹配。然而，与目标人群和LLM训练数据之间关系相关的若干情境因素，可能会影响此类发现的普适性。在本研究中，我们以投票选择为例，调查了LLM能在多大程度上估计德国的公众意见。我们生成了一个合成人物样本，使其与2017年German Longitudinal Election Study受访者的个人特征相匹配。我们要求LLM GPT-3.5预测每位受访者的投票选择，并将这些预测与基于调查的总体和子群体估计进行比较。我们发现，GPT-3.5未能准确预测公民的投票选择，表现出对绿党和左翼党的偏见。尽管LLM捕捉到了“典型”选民子群体（如党派支持者）的倾向，但它遗漏了影响个人投票选择的多方面因素。通过在新的语境下检验基于LLM的投票行为预测，我们的研究为探讨在何种条件下可以利用LLM研究公众意见这一不断发展的领域做出了贡献。研究结果指出了LLM在意见表征上的差异，并强调了将其应用于公众意见估计的局限性。
英文摘要：
The recent development of large language models (LLMs) has spurred discussions about whether LLM-generated "synthetic samples" could complement or replace traditional surveys, considering their training data potentially reflects attitudes and behaviors prevalent in the population. A number of mostly US-based studies have prompted LLMs to mimic survey respondents, with some of them finding that the responses closely match the survey data. However, several contextual factors related to the relationship between the respective target population and LLM training data might affect the generalizability of such findings. In this study, we investigate the extent to which LLMs can estimate public opinion in Germany, using the example of vote choice. We generate a synthetic sample of personas matching the individual characteristics of the 2017 German Longitudinal Election Study respondents. We ask the LLM GPT-3.5 to predict each respondent's vote choice and compare these predictions to the survey-based estimates on the aggregate and subgroup levels. We find that GPT-3.5 does not predict citizens' vote choice accurately, exhibiting a bias towards the Green and Left parties. While the LLM captures the tendencies of "typical" voter subgroups, such as partisans, it misses the multifaceted factors swaying individual voter choices. By examining the LLM-based prediction of voting behavior in a new context, our study contributes to the growing body of research about the conditions under which LLMs can be leveraged for studying public opinion. The findings point to disparities in opinion representation in LLMs and underscore the limitations in applying them for public opinion estimation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
