arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.02426stat.MLcs.LG
收藏

通过适配XGBoost的单调树基GAMI模型

Monotone Tree-Based GAMI Models by Adapting XGBoost

Corporate Model Risk, Wells Fargo(富国银行企业模型风险部门)

Linwei Hu, Soroush Aramideh, Jie Chen, Vijayan N. Nair

AI总结:

本文通过适配XGBoost算法开发单调树基GAMI模型(monotone GAMI-Tree),采用过滤重要交互、拟合单调模型并解析净化结果的方法,实现可解释的单调函数ANOVA建模。

摘要:

近期的论文使用机器学习架构来拟合具有主效应和二阶交互作用的低阶函数ANOVA模型。这些GAMI(GAM+交互)模型是直接可解释的,因为函数主效应和交互作用可以轻松绘制和可视化。不幸的是,将单调性要求纳入基于提升树的现有GAMI模型(如EBM(Lou等,2013)和GAMI-Lin-T(Hu等,2022))并不容易。本文考虑形式为$f(x)=\sum_{j,k}f_{j,k}(x_j, x_k)$的模型,并通过适配XGBoost算法开发了单调树基GAMI模型,称为单调GAMI-Tree。使用XGBoost中的选项直接拟合$f(x)$的单调模型是简单的。然而,拟合的模型仍然是黑盒。我们采取不同的方法:i)使用过滤技术确定重要的交互作用,ii)使用选定的交互作用拟合单调XGBoost算法,最后iii)解析并净化结果以获得单调GAMI模型。使用模拟数据集来展示mono-GAMI-Tree和EBM的行为,两者都使用分段常数拟合。注意,单调性要求是针对完整模型的。在某些情况下,主效应也将是单调的。但是,如示例所示,交互作用不会是单调的。

英文摘要:

Recent papers have used machine learning architecture to fit low-order functional ANOVA models with main effects and second-order interactions. These GAMI (GAM + Interaction) models are directly interpretable as the functional main effects and interactions can be easily plotted and visualized. Unfortunately, it is not easy to incorporate the monotonicity requirement into the existing GAMI models based on boosted trees, such as EBM (Lou et al. 2013) and GAMI-Lin-T (Hu et al. 2022). This paper considers models of the form $f(x)=\sum_{j,k}f_{j,k}(x_j, x_k)$ and develops monotone tree-based GAMI models, called monotone GAMI-Tree, by adapting the XGBoost algorithm. It is straightforward to fit a monotone model to $f(x)$ using the options in XGBoost. However, the fitted model is still a black box. We take a different approach: i) use a filtering technique to determine the important interactions, ii) fit a monotone XGBoost algorithm with the selected interactions, and finally iii) parse and purify the results to get a monotone GAMI model. Simulated datasets are used to demonstrate the behaviors of mono-GAMI-Tree and EBM, both of which use piecewise constant fits. Note that the monotonicity requirement is for the full model. Under certain situations, the main effects will also be monotone. But, as seen in the examples, the interactions will not be monotone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑