arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.06428cs.LG
收藏

部分可解释决策树

Partially Interpretable Decision Trees

University of Amsterdam(阿姆斯特丹大学)

Ludovic Schwartz, Federico Berton, Tim van Erven

AI总结:

本文提出三种部分可解释决策树算法,在真实数据集上性能相近,在黑盒准确率不均的合成数据集上,延迟感知广义CART优于常规CART,可替代概念瓶颈模型后的不可解释神经网络。

摘要:

部分可解释模型通过将复杂输入交由准确但不可解释的黑盒模型处理,扩展了简单可解释模型的能力。延迟处理更多输入可提升模型准确率,延迟处理更少输入则增强可解释性。现有训练此类部分可解释模型的方法基于线性模型或决策规则,本文提出三种针对决策树的算法:现有Branches算法的部分可解释扩展(该算法执行精确搜索以优化树结构);在标准CART之上运行的转换算法(计算效率更高);以及一种广义CART,其分裂准则在树生长过程中会考虑延迟处理选项。令人惊讶的是,在多个真实世界数据集上,三种算法的性能表现相似,这可通过对智能树构建潜在收益的形式化边界来解释,前提是黑盒准确率在整个域内足够均匀。我们还构建了一个合成数据集,其中黑盒准确率故意不均匀,此时我们的延迟感知广义CART确实优于常规CART,且与精确搜索的性能相当。最后,我们评估了其在概念瓶颈模型中的应用:部分可解释决策树可替代瓶颈后的不可解释神经网络,在未延迟处理的输入上,该模型完全可解释,且延迟处理能保持准确率。

英文摘要:

Partially interpretable models extend simple interpretable models with the option to defer complicated inputs to an accurate but uninterpretable black-box model. By deferring more inputs, the model gets more accurate; by deferring fewer, it gets more interpretable. Existing procedures to train such partially interpretable models build on linear models or decision rules. We propose three algorithms for decision trees: a partially interpretable extension of the existing Branches algorithm, which performs exact search to optimize the tree; a conversion algorithm that we run on top of standard CART, which is computationally more efficient; and a generalized CART whose splitting criterion takes into account the option to defer while growing the tree. Perhaps surprisingly, the performance of the three algorithms is similar across many realworld datasets. We can explain this through a formal bound on the potential gain of smart tree construction, provided the black box accuracy is sufficiently homogeneous across the domain. We also construct a synthetic dataset where the accuracy of the black box is deliberately inhomogeneous. There, our deferral-aware generalization of CART does outperform regular CART and matches the performance of exact search. Finally, we evaluate an application to concept bottleneck models. Partially interpretable decision trees can replace uninterpretable neural networks after the bottleneck. On undeferred inputs, the model is fully interpretable, and deferring preserves accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑