本文提出 I 类错误率通常不会被夸大,并通过多种研究实践说明可重复性危机更多源于研究者混淆统计错误与理论错误。
摘要:
I 类错误率的夸大被认为是可重复性危机的成因之一。诸如 p 值操纵等有问题的研究实践被认为会将 I 类错误率夸大至其名义水平之上,从而导致文献中出现意外偏高的假阳性水平,并进而导致意外偏低的重复率。在本文中,我提出一种不同的观点。我认为,有问题的研究实践及其他研究实践通常并不会夸大相关的 I 类错误率。我首先引入 I 类错误率的概念,并区分统计错误与理论错误。随后,我针对模型误设、多重检验、选择性推断、分叉路径、探索性分析、p 值操纵、可选停止、双重使用数据以及 HARKing 等情况阐述我的论点。在每一种情形中,我都证明相关的 I 类错误率通常不会被夸大至其名义水平之上,而在少数被夸大的情况下,这种夸大也很容易被识别并解决。我的结论是,可重复性危机至少可以部分地由研究者对统计错误的误读以及对理论错误的低估来解释。
英文摘要:
The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view. I argue that questionable and other research practices do not usually inflate relevant Type I error rates. I begin by introducing the concept of Type I error rates and distinguishing between statistical errors and theoretical errors. I then illustrate my argument with respect to model misspecification, multiple testing, selective inference, forking paths, exploratory analyses, p-hacking, optional stopping, double dipping, and HARKing. In each case, I demonstrate that relevant Type I error rates are not usually inflated above their nominal level, and in the rare cases that they are, the inflation is easily identified and resolved. I conclude that the replication crisis may be explained, at least in part, by researchers' misinterpretation of statistical errors and their underestimation of theoretical errors.
Correcting Measurement Error and Zero Inflation in Functional Covariates for Scalar-on-Function Quantile Regression
Indiana University School of Public Health(印第安纳大学公共卫生学院);Oregon State University(俄勒冈州立大学);Marmara University(马尔马拉大学);Texas A&M University School of Public Health(德克萨斯农工大学公共卫生学院);Columbia University Mailman School of Public Health(哥伦比亚大学米尔曼公共卫生学院)
Addressing zero-inflated and mis-measured functional predictors in scalar-on-function regression model
Oregon State University College of Science(俄勒冈州立大学理学院);Marmara University Faculty of Sciences(马尔马拉大学理学院);Columbia University Mailman School of Public Health(哥伦比亚大学梅尔曼公共卫生学院);Texas A&M University School of Public Health(德克萨斯农工大学公共卫生学院)
Testing for Publication Bias in Diagnostic Meta-Analysis: A Simulation Study
本研究通过模拟的方法研究了几种统计检验在诊断性荟萃分析中检测发表偏倚的性能。虽然双变量模型应用于汇总诊断荟萃分析中主要研究的数据,但诊断准确性的单变量指标更适合检测发表偏倚。与早期的研究相反,早期的研究仅关注诊断比值比或其对数($\lnω$),这些测试与四种不同的诊断准确性单变量测量相结合。对于检验和单变量测量的每种组合,在不同条件下检查第一类错误率和统计功效。结果表明,在诊断荟萃分析中,不推荐使用基于线性回归或等级相关的检验,因为无论采用何种单变量测量,第一类错误率都被夸大或把握度太低。相比之下,修整和填充与$\lnω$的组合即使在极端情况下(至少当每项荟萃分析的研究数量足够大时),也具有非膨胀或仅轻微膨胀的第一类错误率和中到高功效。因此,我们建议在诊断性荟萃分析中应用修剪和填充结合$\lnω$检测漏斗图不对称性。请引用发表在Statistics in Medicine(https://doi.org/10.1002/sim.6177)上的这篇论文。
Minor Issues Escalated to Critical Levels in Large Samples: A Permutation-Based Fix
Department of Mathematics and Statistics, University of Victoria(维多利亚大学数学与统计系);Department of Mathematics and Statistics, University of Saskatchewan(萨斯喀彻温大学数学与统计系)