为可解释性拆解树集成：一种层次化可视化工具与一棵多变量最优重建树
Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree
AI总结：
针对树集成模型的黑箱问题，本文结合特征使用层次热图与带稀疏惩罚的 MILP 最优多变量代理树，实现对其预测行为的可解释近似。
摘要：
由于算法决策对现实世界应用的影响不断增大，模型的可解释性已成为机器学习中的一个关键问题。树集成方法（如 Random Forests 或 XgBoost）是用于分类任务的强大学习工具。然而，尽管组合多棵树可能比单棵树提供更高的预测质量，但这会牺牲可解释性，从而产生“黑箱”模型。鉴于此，我们旨在开发一种树集成模型的可解释表示，以便为理解其行为提供有价值的见解。首先，在给定目标树集成模型的情况下，我们基于森林特征使用情况的热图表示开发了一种层次化可视化工具，其中将特征的频率及其被选择的层级作为重要性指标。接着，我们提出了一种混合整数线性规划（MILP）公式，用于构建一棵能够准确模拟目标模型预测的单一最优多变量树。其目标是提供一种基于斜超平面分裂的可解释代理模型，该模型根据所定义的森林重要性指标仅使用最相关的特征。该 MILP 模型根据特征在森林中的出现频率对特征选择施加惩罚，以进一步促使分裂具有稀疏性。我们还强化了这一自然公式，以提升混合整数软件的计算性能。使用最先进的现成求解器，在来自 UCI 存储库的基准数据集上进行了计算实验。结果表明，所提出模型能够有效生成一棵浅层可解释树，用于近似树集成的决策函数。
英文摘要：
The interpretability of models has become a crucial issue in Machine Learning because of algorithmic decisions' growing impact on real-world applications. Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning tools for classification tasks. However, while combining multiple trees may provide higher prediction quality than a single one, it sacrifices the interpretability property resulting in "black-box" models. In light of this, we aim to develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. First, given a target tree-ensemble model, we develop a hierarchical visualization tool based on a heatmap representation of the forest's feature use, considering the frequency of a feature and the level at which it is selected as an indicator of importance. Next, we propose a mixed-integer linear programming (MILP) formulation for constructing a single optimal multivariate tree that accurately mimics the target model predictions. The goal is to provide an interpretable surrogate model based on oblique hyperplane splits, which uses only the most relevant features according to the defined forest's importance indicators. The MILP model includes a penalty on feature selection based on their frequency in the forest to further induce sparsity of the splits. The natural formulation has been strengthened to improve the computational performance of {mixed-integer} software. Computational experience is carried out on benchmark datasets from the UCI repository using a state-of-the-art off-the-shelf solver. Results show that the proposed model is effective in yielding a shallow interpretable tree approximating the tree-ensemble decision function.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
