arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.14526stat.ME
收藏

政治学中回归不连续设计对已发表研究结果的可靠性

On the reliability of published findings using the regression discontinuity design in political science

Drew Stommes, P. M. Aronow, Fredrik Sävje

摘要:

不连续回归设计在弱假设条件下提供了因果效应的识别,使其成为现代政治学研究的标准方法。但识别并不一定意味着因果效应可以用有限的数据准确估计。在本文中,我们强调,估计下的RD设计涉及严重的统计挑战,并探讨这些挑战如何体现在政治学的实证文献。我们收集了2009年至2018年期间发表在顶级政治科学期刊上的所有基于RD的研究结果。已发表结果的分布呈现出病态特征;估计值往往集中在传统的统计显著性水平之上。对所有研究的重新分析表明,研究人员的自由裁量权不是这些功能的主要驱动力。然而,研究人员倾向于使用不适当的方法进行推断,人为地使标准误变小。回顾性功效分析显示,这些研究中的大多数都没有足够的功效来检测除了大效应之外的所有效应。我们发现的问题,结合学术出版中有据可查的选择压力,引起了人们的担忧,即许多使用RD设计的已发表研究结果可能被夸大了。

英文摘要:

The regression discontinuity (RD) design offers identification of causal effects under weak assumptions, earning it a position as a standard method in modern political science research. But identification does not necessarily imply that causal effects can be estimated accurately with limited data. In this paper, we highlight that estimation under the RD design involves serious statistical challenges and investigate how these challenges manifest themselves in the empirical literature in political science. We collect all RD-based findings published in top political science journals in the period 2009-2018. The distribution of published results exhibits pathological features; estimates tend to bunch just above the conventional level of statistical significance. A reanalysis of all studies with available data suggests that researcher discretion is not a major driver of these features. However, researchers tend to use inappropriate methods for inference, rendering standard errors artificially small. A retrospective power analysis reveals that most of these studies were underpowered to detect all but large effects. The issues we uncover, combined with well-documented selection pressures in academic publishing, cause concern that many published findings using the RD design may be exaggerated.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑