arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.02306stat.APstat.ME
收藏

事后选择显著性水平α:多重标准显著性阈值的危险

Choosing alpha post hoc: the danger of multiple standard significance thresholds

Jesse Hemerik, Nick W Koning

AI总结:

本文指出将显著性阈值α从0.05降至0.005可能导致同一领域出现多重标准阈值,使研究者事后根据p值选择α,从而严重破坏假设检验有效性,并讨论了可能的解决方案。

摘要:

经典假设检验的一个基本假设是显著性阈值α独立于数据被选择。置信区间的有效性同样依赖于事先选择α。我们指出,α的独立性在实践中得到保证,因为在大多数领域中,存在一个所有人共同使用的标准α——因此α自动独立于一切。然而,最近有人呼吁将α从0.05降低到0.005。我们注意到,这可能导致在同一个科学领域内出现多个被接受的标准阈值。例如,不同期刊可能要求不同的显著性阈值。其后果是,一些研究者可能受到诱惑,根据自己的p值方便地选择α。我们用例子说明这会严重破坏假设检验的有效性,并提及一些可能的解决方案。

英文摘要:

A fundamental assumption of classical hypothesis testing is that the significance threshold $α$ is chosen independently from the data. The validity of confidence intervals likewise relies on choosing $α$ beforehand. We point out that the independence of $α$ is guaranteed in practice because, in most fields, there exists one standard $α$ that everyone uses -- so that $α$ is automatically independent of everything. However, there have been recent calls to decrease $α$ from $0.05$ to $0.005$. We note that this may lead to multiple accepted standard thresholds within one scientific field. For example, different journals may require different significance thresholds. As a consequence, some researchers may be tempted to conveniently choose their $α$ based on their p-value. We use examples to illustrate that this severely invalidates hypothesis tests, and mention some potential solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑