ChatGPT vs Social Surveys：探究客观与主观的硅基人口
ChatGPT vs Social Surveys: Probing Objective and Subjective Silicon Population
AI总结：
本文通过重复随机抽样探究GPT生成的硅基人口特征，发现其在性别和年龄上与美国人口相符，但高估了黑人和高学历人群，且态度估计不一致、响应更具确定性，揭示了其对现实人口推断的潜在风险。
摘要：
近期关于大型语言模型（LLMs）的讨论表明，它们具有在社会调查中模拟人类回答并生成可靠预测的潜力，例如在政治民调中的预测。然而，现有研究结果高度不一致，使我们对LLMs生成数据的人口特征感到不确定。本文采用重复随机抽样方法创建抽样分布，以识别由GPT生成的硅基样本的总体参数。研究结果表明，GPT的人口统计分布在性别和平均年龄方面与2020年美国人口相符。然而，即使拥有准确的知识，GPT仍显著高估了黑人群体和受过高等教育人群的代表性。此外，GPT对态度得分的点估计高度不一致，且未表现出对任何特定意识形态的明显倾向。样本响应分布呈现正态模式，与人类受访者的分布存在显著差异。与先前研究一致，我们发现GPT的回答比人类更具确定性。最后，我们讨论了这种存在偏见且确定性的硅基人口对现实世界人口进行推断所带来的令人担忧的影响。
英文摘要：
Recent discussions about Large Language Models (LLMs) indicate that they have the potential to simulate human responses in social surveys and generate reliable predictions, such as those found in political polls. However, the existing findings are highly inconsistent, leaving us uncertain about the population characteristics of data generated by LLMs. In this paper, we employ repeated random sampling to create sampling distributions that identify the population parameters of silicon samples generated by GPT. Our findings show that GPT's demographic distribution aligns with the 2020 U.S. population in terms of gender and average age. However, GPT significantly overestimates the representation of the Black population and individuals with higher levels of education, even when it possesses accurate knowledge. Furthermore, GPT's point estimates for attitudinal scores are highly inconsistent and show no clear inclination toward any particular ideology. The sample response distributions exhibit a normal pattern that diverges significantly from those of human respondents. Consistent with previous studies, we find that GPT's answers are more deterministic than those of humans. We conclude by discussing the concerning implications of this biased and deterministic silicon population for making inferences about real-world populations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
