一生万物:使用语言模型模拟人类样本
Out of One, Many: Using Language Models to Simulate Human Samples
AI总结:
本文提出利用GPT-3语言模型通过社会人口背景条件设定生成“硅样本”,验证其具备细粒度且与人口特征相关的“算法保真度”,可作为社会科学研究中模拟特定人类子群体的新型工具。
摘要:
我们提出并探讨了一种可能性,即在社会科学研究中,语言模型可以作为特定人类子群体的有效代理加以研究。人工智能工具的实际应用和研究应用有时会受到问题性偏差(如种族主义或性别歧视)的限制,而这些偏差通常被视为模型的统一属性。我们表明,在其中一个工具——GPT-3语言模型——内部的“算法偏差”实际上是细粒度的,并且与人口统计学特征相关,这意味着适当的条件设定将使其能够准确模拟来自各种人类子群体的回答分布。我们将这一特性称为“算法保真度”,并探讨其在GPT-3中的表现程度。我们通过以美国进行的多项大型调查中数千名真实人类参与者的社会人口背景故事为条件对模型进行设定,从而创建“硅样本”。随后,我们比较硅样本与人类样本,以证明GPT-3中所包含的信息远超表面相似性。它是细致入微、多层面的,并反映了表征人类态度的观念、态度与社会文化背景之间的复杂相互作用。我们认为,具有足够算法保真度的语言模型因此构成了一种新颖而强大的工具,可推动跨多个学科对人类与社会的理解。
英文摘要:
We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the "algorithmic bias" within one such tool -- the GPT-3 language model -- is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property "algorithmic fidelity" and explore its extent in GPT-3. We create "silicon samples" by conditioning the model on thousands of socio-demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT-3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio-cultural context that characterize human attitudes. We suggest that language models with sufficient algorithmic fidelity thus constitute a novel and powerful tool to advance understanding of humans and society across a variety of disciplines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。