With limited resources, scientific inquiries must be prioritized for further study, funding, and translation based on their practical significance: whether the effect size is large enough to be meaningful in the real world. Doing so must evaluate a result's effect strength, defined as a conservative assessment of practical significance. We propose the least difference in means ($δ_L$) as a two-sample statistic that can quantify effect strength and perform a hypothesis test to determine if a result has a meaningful effect size. To facilitate consensus, $δ_L$ allows scientists to compare effect strength between related results and choose different thresholds for hypothesis testing without recalculation. Both $δ_L$ and the relative $δ_L$ outperform other candidate statistics in identifying results with higher effect strength. We use real data to demonstrate how the relative $δ_L$ compares effect strength across broadly related experiments. The relative $δ_L$ can prioritize research based on the strength of their results.
A Formal Graphical Inference Framework for Combining Effect Size with Statistical Significance: Application to Multivariate and Functional Linear Models
Faculty of Agriculture and Technology, University of South Bohemia(南波希米亚大学农业与技术学院);Natural Resources Institute Finland (Luke)(芬兰自然资源研究所(Luke));University of Helsinki(赫尔辛基大学)
Statistical Significance of Feature Importance Rankings
Department of Statistics University of California, Berkeley(统计学系加州大学伯克利分校);Department of Statistics and Data Science University of Pennsylvania(统计学与数据科学系宾夕法尼亚大学)
本文针对特征重要性排名因随机采样而不稳定的问题,提出基于假设检验的验证方法和两种高效采样算法,能以超过 1-α 的概率正确识别前 K 个重要特征及其顺序,并在 SHAP 和 LIME 上验证了有效性。
Pathwise Least Angle Regression and a Significance Test for the Elastic Net
Efron等人(2004)提出的最小角回归(LARS)是一种构造Lasso解的分段线性路径的新方法。几年来,它仍然是事实上的方法计算Lasso解决方案之前,更复杂的优化算法before it. LARS方法最近再次增加了它的普及,由于它能够找到的值的惩罚参数,称为结,在一个新的参数进入非零系数的活动集。例如,Lockhart等人(2014)对Lasso的显著性检验需要通过LARS算法求解结。另一方面,弹性网络(EN)是Lasso的一个非常流行的扩展,它使用Lasso和岭回归惩罚的线性组合。在本文中,我们提出了一种新的算法,称为路径(PW-)LARS-EN,这是能够计算EN调谐参数α值的网格上的EN节点。所开发的PW-LARS-EN算法减少了EN调谐参数,并利用以前找到的结值和原始LARS算法。然后将Lasso的协方差检验统计量推广到EN,以检验预测因子的显著性。我们的模拟研究验证了检验统计量具有渐近Exp(1)分布的事实。
A significance test for forward stepwise model selection
我们将Lockhart et al.(2013)和Taylor et al.(2013)开发的惩罚回归显著性检验方法应用于前向逐步模型选择。二次不等式描述的选择过程的一般框架包括一个变量的逐步向前分组变量,使我们能够处理分类变量和因子模型。我们提供了一个算法来计算一个新的统计量与一个确切的零分布条件的模型选择过程的结果。这个新的统计量,我们表示为$Tχ$,在全局空值下具有截断的$χ$分布。我们在每一步之后的残差上逐步迭代地向前应用该测试。所得到的方法具有逐步选择的计算优势,并解决了由于模型选择而导致的无效检验统计量的问题。我们说明了这种方法的灵活性,通过将其应用到几个专业的应用程序,包括一个分层的相互作用模型和最近描述的添加剂模型,自适应地选择线性和非线性效应之间的每个变量。