Moving beyond the classic difference-in-differences model: A simulation study comparing statistical methods for estimating effectiveness of state-level policies
Beth Ann Griffin, Megan S. Schuler, Elizabeth A. Stuart, Stephen Patrick, Elizabeth McNeer, Rosanna Smart, David Powell, Bradley D. Stein, Terry Schell, Rosalie L. Pacula
State-level policy evaluations commonly employ a difference-in-differences (DID) study design; yet within this framework, statistical model specification varies notably across studies. Motivated by applied state-level opioid policy evaluations, this simulation study compares statistical performance of multiple variations of two-way fixed effect models traditionally used for DID under a range of simulation conditions. While most linear models resulted in minimal bias, non-linear models and population-weighted versions of classic linear two-way fixed effect and linear GEE models yielded considerable bias (60 to 160%). Further, root mean square error is minimized by linear AR models when examining crude mortality rates and by negative binomial models when examining raw death counts. In the context of frequentist hypothesis testing, many models yielded high Type I error rates and very low rates of correctly rejecting the null hypothesis (< 10%), raising concerns of spurious conclusions about policy effectiveness. When considering performance across models, the linear autoregressive models were optimal in terms of directional bias, root mean squared error, Type I error, and correct rejection rates. These findings highlight notable limitations of traditional statistical models commonly used for DID designs, designs widely used in opioid policy studies and in state policy evaluations more broadly.
From Observational Data to Clinical Recommendations: A Causal Framework for Estimating Patient-level Treatment Effects and Learning Policies
Faculty of Data and Decision Sciences, Technion（数据与决策科学学院，技术ion）；Blavatnik School of Computer Science, Tel-Aviv University（Blavatnik计算机科学学院，特拉维夫大学）；Department of Cardiology, Rambam Health Care Campus（心内科部，拉马姆医疗中心）；The Bruce Rappaport Faculty of Medicine, Technion（布鲁斯·拉帕波特医学院，技术ion）；Department of Statistics and Operations Research, School of Mathematical Sciences, Tel Aviv University（统计与运筹学部，数学科学学院，特拉维夫大学）
Modified treatment policies that depend on the natural history of treatment
New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）；University of California at Berkeley（加州大学伯克利分校）；University of Copenhagen（哥本哈根大学）；Columbia University（哥伦比亚大学）
