一种基于设计的文本因果推断解决方案：语言模型会不会太大？
A Design-based Solution for Causal Inference with Text: Can a Language Model Be Too Large?
AI总结：
针对文本因果推断中的潜在混杂和重叠偏差，提出一种新的实验设计，并实证发现LLM方法不如词袋模型，同时揭示了政治传播中谦逊表达的因果说服效应。
摘要：
许多社会科学问题探讨语言属性如何因果性地影响受众的态度和行为。由于文本属性往往相互关联（例如，愤怒的评论使用粗俗语言），我们必须控制可能的潜在混杂因素以隔离因果效应。近期文献提出改编大型语言模型（LLMs）来学习文本的潜在表示，这些表示能成功预测处理变量和结果。然而，由于处理变量是文本的一个组成部分，这些深度学习方法有风险学习到实际上编码了处理变量本身的表示，从而引发重叠偏差。我们不依赖于事后调整，而是引入一种新的实验设计，该设计能处理潜在混杂、避免重叠问题，并无偏地估计处理效应。我们将此设计应用于一项评估政治传播中表达谦逊的说服力的实验。在方法上，我们证明基于LLM的方法在使用我们的真实文本和实验结果的条件下，其表现甚至不如简单的词袋模型。在实质上，我们隔离了表达谦逊对政治陈述感知说服力的因果效应，为社交媒体平台、政策制定者和社会科学家提供了关于传播效应的新见解。
英文摘要：
Many social science questions ask how linguistic properties causally affect an audience's attitudes and behaviors. Because text properties are often interlinked (e.g., angry reviews use profane language), we must control for possible latent confounding to isolate causal effects. Recent literature proposes adapting large language models (LLMs) to learn latent representations of text that successfully predict both treatment and the outcome. However, because the treatment is a component of the text, these deep learning methods risk learning representations that actually encode the treatment itself, inducing overlap bias. Rather than depending on post-hoc adjustments, we introduce a new experimental design that handles latent confounding, avoids the overlap issue, and unbiasedly estimates treatment effects. We apply this design in an experiment evaluating the persuasiveness of expressing humility in political communication. Methodologically, we demonstrate that LLM-based methods perform worse than even simple bag-of-words models using our real text and outcomes from our experiment. Substantively, we isolate the causal effect of expressing humility on the perceived persuasiveness of political statements, offering new insights on communication effects for social media platforms, policy makers, and social scientists.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
