arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.22346cs.LGcs.AI
收藏

梯度提升解释中的首 mover 偏差:机制、检测与解决

First-Mover Bias in Gradient Boosting Explanations: Mechanism, Detection, and Resolution

Drake Caraker, Bryan Arnold, David Rhoads

AI总结:

本文探讨了梯度提升中SHAP特征重要性因序列残差拟合导致的首 mover 偏差机制,提出DASH和随机重训练方法可缓解该问题,提升模型稳定性。

摘要:

我们识别出首 mover 偏差——在梯度提升中由于序列残差拟合导致的SHAP特征重要性集中现象——作为多共线性下归因不稳定的机制性因素。扩大单个模型规模会加剧这一效应:一个大型单模型在匹配我们方法的总树数量时,其归因可重复性最差。我们证明模型独立性在很大程度上中和了首 mover 偏差。DASH(多样化SHAP聚合)和简单的种子平均(随机重训练)通过打破序列依赖链恢复稳定性。在rho=0.9时,两者达到约0.977的稳定性,而单最佳模型降为0.958,LSM降为0.938。在乳腺癌数据上,DASH将稳定性从0.376提升到0.925(+0.549),优于随机重训练0.063。在非线性DGPs下,优势在rho≥0.7时显现。DASH提供两个诊断工具——特征稳定性指数和重要性-稳定性图——可检测首 mover 偏差而无需真实数据。交叉方差分析与正式F统计量确认了机制:DASH将方差从模型主导(40.6%)转移到数据主导(73.6%)。软件可在https://github.com/DrakeCaraker/dash-shap获取。

英文摘要:

We identify first-mover bias -- path-dependent concentration of SHAP feature importance from sequential residual fitting in gradient boosting -- as a mechanistic contributor to attribution instability under multicollinearity. Scaling up a single model amplifies this effect: a Large Single Model matching our method's total tree count produces the poorest attribution reproducibility of any approach tested. We show that model independence largely neutralizes first-mover bias. Both DASH (Diversified Aggregation of SHAP) and simple seed-averaging (Stochastic Retrain) restore stability by breaking the sequential dependency chain. At rho=0.9, both achieve stability ~0.977, while Single Best degrades to 0.958 and LSM to 0.938. On Breast Cancer, DASH improves stability from 0.376 to 0.925 (+0.549), outperforming Stochastic Retrain by +0.063. Under nonlinear DGPs, the advantage emerges at rho>=0.7. DASH provides two diagnostic tools -- the Feature Stability Index and Importance-Stability Plot -- that detect first-mover bias without ground truth. A crossed ANOVA with formal F-statistics confirms the mechanism: DASH shifts variance from model-dominated (40.6%) to data-dominated (73.6%). Software at https://github.com/DrakeCaraker/dash-shap

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑