arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.07293stat.MLcs.LG
收藏

具有理论保证的不确定性非线性变量选择的统一框架

Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees

Harvard University(哈佛大学);Google Research(谷歌研究院)

Wenying Deng, Beau Coker, Rajarshi Mukherjee, Jeremiah Zhe Liu, Brent A. Coull

摘要:

我们为非线性变量选择开发了一个简单而统一的框架,该框架将不确定性纳入预测函数,并与各种机器学习模型(例如,树集合、核方法、神经网络等)。特别是,对于学习的非线性模型$f(\mathbf{x})$,我们考虑使用积分偏导数$Ψ_j = \Vert \frac{\partial}{\partial \mathbf{x}^j} f(\mathbf{x})\Vert^2_{P_\mathcal{X}}$量化输入变量$\mathbf{x}^j$的重要性。然后,我们(1)提供了一个原则性的方法来量化变量选择的不确定性,通过推导其后验分布,(2)表明,该方法是可推广的,甚至不可微的模型,如树集成。严格的贝叶斯非参数定理,以保证所提出的方法的后验一致性和渐近不确定性。在医疗基准数据集上的大量仿真和实验证实了该算法优于现有的经典和最近的变量选择方法。

英文摘要:

We develop a simple and unified framework for nonlinear variable selection that incorporates uncertainty in the prediction function and is compatible with a wide range of machine learning models (e.g., tree ensembles, kernel methods, neural networks, etc). In particular, for a learned nonlinear model $f(\mathbf{x})$, we consider quantifying the importance of an input variable $\mathbf{x}^j$ using the integrated partial derivative $Ψ_j = \Vert \frac{\partial}{\partial \mathbf{x}^j} f(\mathbf{x})\Vert^2_{P_\mathcal{X}}$. We then (1) provide a principled approach for quantifying variable selection uncertainty by deriving its posterior distribution, and (2) show that the approach is generalizable even to non-differentiable models such as tree ensembles. Rigorous Bayesian nonparametric theorems are derived to guarantee the posterior consistency and asymptotic uncertainty of the proposed approach. Extensive simulations and experiments on healthcare benchmark datasets confirm that the proposed algorithm outperforms existing classic and recent variable selection methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑