We study issues related to external validity for treatment effects using over 100 replications of the Angrist and Evans (1998) natural experiment on the effects of sibling sex composition on fertility and labor supply. The replications are based on census data from around the world going back to 1960. We decompose sources of error in predicting treatment effects in external contexts in terms of macro and micro sources of variation. In our empirical setting, we find that macro covariates dominate over micro covariates for reducing errors in predicting treatments, an issue that past studies of external validity have been unable to evaluate. We develop methods for two applications to evidence-based decision-making, including determining where to locate an experiment and whether policy-makers should commission new experiments or rely on an existing evidence base for making a policy decision.
Principal Stratification with Bayesian Additive Regression Trees for Count-Valued Intermediate Variables: Estimating the Effect of Fertility on Women's Employment
On the probability of invalidating a causal inference due to limited external validity
在实证研究中，特别是在随机实验中，外部效度往往受到质疑，因为内部效度和外部效度之间存在权衡。为了量化外部效度的稳健性，必须首先概念化完全代表目标人群的样本之间的差距（即，理想样本）和观察样本。借鉴Frank & Min（2007）和Frank et al.（2013），我将这种差距定义为未观察到的样本，并打算在本研究中量化其与原假设统计检验（NHST）的关系。由于有限的外部有效性而使因果推断无效的概率，即，PEV是基于理想样本未能拒绝原假设的概率，前提是原假设已基于观察样本被拒绝。本研究通过一个实证例子说明了用PEV评价外部效度的指导思想和程序（即，Borman等人（2008））。具体而言，人们将能够定位未观察到的样本统计量的阈值，该阈值将使PEV高于期望值，并使用该信息来表征未观察到的样本，该未观察到的样本将使所讨论的研究的外部有效性不那么稳健。当NHST被认为是基于理想样本时，PEV被证明与统计功效相关。
Using a Two-Parameter Sensitivity Analysis Framework to Efficiently Combine Randomized and Non-randomized Studies
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California Davis（加州大学戴维斯分校）；University of California San Francisco（加州大学旧金山分校）
Leveraging external data in the analysis of randomized controlled trials: a comparative analysis
Massachusetts General Hospital（麻省总医院）；University of Minnesota School of Public Health（明尼苏达大学公共卫生学院）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
