通过 minP 调整解决研究者自由度问题
Addressing researcher degrees of freedom through minP adjustment
AI总结:
本文将研究者分析选择的多重性建模为多重检验问题,提出基于置换的 minP 调整以处理相关检验,在48种策略中控制假阳性并保留统计功效。
摘要:
当不同研究者使用同一数据集研究同一研究问题时,他们可能得到不同的、甚至潜在相互冲突的结果。这是因为研究者在分析选择上通常具有相当大的灵活性,这一问题也被称为“研究者自由度”。结合选择性报告最小 p 值或最大效应的做法,研究者自由度可能导致假阳性率上升以及结果过于乐观。本文中,我们通过将分析策略的多重性形式化为一个多重检验问题来解决这一问题。由于不同分析策略的检验统计量通常高度相关,Bonferroni 校正等简单方法并不合适,因为它会造成不可接受的功效损失。相反,我们提出使用“minP”调整方法,该方法考虑潜在的检验相关性,并通过基于置换的过程近似最小 p 值的潜在零分布。已知该过程在确保对族系错误率进行弱控制的同时,比更简单的方法获得更高功效。我们通过将其应用于一项关于围手术期 paO2 对神经外科术后并发症影响的研究,来说明这种解决研究者自由度的方法。研究共考虑 48 种分析策略,并使用 minP 过程进行调整。该方法允许选择性地报告产生最有说服力证据的分析策略结果,同时控制第 1 类错误——从而控制发表可能无法复现的假阳性结果的风险。
英文摘要:
When different researchers study the same research question using the same dataset they may obtain different and potentially even conflicting results. This is because there is often substantial flexibility in researchers' analytical choices, an issue also referred to as ''researcher degrees of freedom''. Combined with selective reporting of the smallest p-value or largest effect, researcher degrees of freedom may lead to an increased rate of false positive and overoptimistic results. In this paper, we address this issue by formalizing the multiplicity of analysis strategies as a multiple testing problem. As the test statistics of different analysis strategies are usually highly dependent, a naive approach such as the Bonferroni correction is inappropriate because it leads to an unacceptable loss of power. Instead, we propose using the ''minP'' adjustment method, which takes potential test dependencies into account and approximates the underlying null distribution of the minimal p-value through a permutation-based procedure. This procedure is known to achieve more power than simpler approaches while ensuring a weak control of the family-wise error rate. We illustrate our approach for addressing researcher degrees of freedom by applying it to a study on the impact of perioperative paO2 on post-operative complications after neurosurgery. A total of 48 analysis strategies are considered and adjusted using the minP procedure. This approach allows to selectively report the result of the analysis strategy yielding the most convincing evidence, while controlling the type 1 error -- and thus the risk of publishing false positive results that may not be replicable.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。