In 2015 the Open Science Collaboration (OSC) (Nosek et al 2015) published a highly influential paper which claimed that a large fraction of published results in the psychological sciences were not reproducible. In this article we review this claim from several points of view. We first offer an extended analysis of the methods used in that study. We show that the OSC methodology induces a bias that is able by itself to explain the discrepancy between the OSC estimates of reproducibility and other more optimistic estimates made by similar studies.
The article also offers a more general literature review and discussion of reproducibility in experimental science. We argue, for both scientific and ethical reasons, that a considered balance of false positive and false negative rates is preferable to a single-minded concentration on false positive rates alone.
Why "Redefining Statistical Significance" Will Not Improve Reproducibility and Could Make the Replication Crisis Worse
最近一项“重新定义统计学显著性”的提议(Benjamin等人,Nature Human Behaviour,2017)声称,假阳性率“将立即改善”大于2的因素,复制率只需将“统计学显著性”的传统截止值从P<0.05改变为P<0.005即可翻倍。我分析了这些说法的真实性,特别关注本杰明等人在评估他们的建议的影响时如何忽视了P黑客的影响。我的分析表明,一旦P-hacking被解释为较低阈值的感知益处几乎消失,提示两个主要结论:(i)Benjamin等人(2017)声称的假阳性率和复制率的改善被夸大和误导。(ii)在一些合理的情况下,较低的截止值将使复制危机变得更糟。
A Re-defined and Generalized Percent-Overlap-of-Activation Measure for Studies of fMRI Reproducibility and its Use in Identifying Outlier Activation Maps
A high-reproducibility and high-accuracy method for automated topic classification
Howard Hughes Medical Institute(霍华德·休斯医学研究所);Northwestern University(西北大学);Northwestern University Feinberg School of Medicine(西北大学范伯格医学院);Rehabilitation Institute of Chicago(芝加哥康复研究所)
Synthetic dataset generation methodology for Recommender Systems using statistical sampling methods, a Multinomial Logit model, and a Fuzzy Inference System
Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease
Biomedical Informatics & Data Science, Johns Hopkins University(约翰霍普金斯大学生物医学信息学与数据科学);Biomedical & Health Informatics, University of Washington(华盛顿大学生物医学与健康信息学)
Reproducibility Assessment of Magnetic Resonance Spectroscopy of Pregenual Anterior Cingulate Cortex across Sessions and Vendors via the Cloud Computing Platform CloudBrain-MRS
National Institute for Data Science in Health and Medicine, Xiamen University(厦门大学健康与医学数据科学国家研究院);College of Ocean & Earth Sciences, Xiamen University(厦门大学海洋与地球学院);Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance, Xiamen University(厦门大学福建省等离子体与磁共振重点实验室);Department of Electronic Science, Xiamen University(厦门大学电子科学系);Clinical and Technical Support, Philips Healthcare(飞利浦医疗保健临床与技术支持部);School of Economics and Management, Xiamen University of Technology(厦门理工学院经济与管理学院);Department of Radiology, the First Hospital Affiliated to Xiamen University(厦门大学附属第一医院放射科);China Mobile (Suzhou) Software Technology Company Limited(中国移动苏州软件技术有限公司);School of Computer and Information Engineering, Xiamen University of Technology(厦门理工学院计算机与信息工程学院)
A statistical methodology to select covariates in high-dimensional data under dependence. Application to the classification of genetic profiles in oncology