arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.08690cs.LGstat.ML
收藏

广义可扩展的最优稀疏决策树

Generalized and Scalable Optimal Sparse Decision Trees

University of British Columbia(不列颠哥伦比亚大学);Duke University(杜克大学)

Jimmy Lin, Chudi Zhong, Diane Hu, Cynthia Rudin, Margo Seltzer

摘要:

从计算的角度来看,决策树优化是非常困难的,但对于可解释的机器学习领域来说是必不可少的。尽管在过去的40年里一直在努力,但直到最近才取得了优化突破,使实用算法能够找到最佳决策树。这些新技术有可能引发范式转变,其中可以构建稀疏决策树来有效地优化各种目标函数,而不依赖于贪婪分裂和修剪算法,这些算法通常会导致次优解。在这项工作中的贡献是提供一个通用的框架决策树优化,解决了两个重要的开放问题,在该地区:不平衡数据的处理和充分优化连续变量。我们提出的技术,产生最佳的决策树在各种目标,包括F-分数,AUC,和部分区域下的ROC凸包。我们还介绍了一个可扩展的算法,产生可证明的最佳结果,在连续变量的存在下,并加快了决策树建设的几个数量级相对于最先进的。

英文摘要:

Decision tree optimization is notoriously difficult from a computational perspective but essential for the field of interpretable machine learning. Despite efforts over the past 40 years, only recently have optimization breakthroughs been made that have allowed practical algorithms to find optimal decision trees. These new techniques have the potential to trigger a paradigm shift where it is possible to construct sparse decision trees to efficiently optimize a variety of objective functions without relying on greedy splitting and pruning heuristics that often lead to suboptimal solutions. The contribution in this work is to provide a general framework for decision tree optimization that addresses the two significant open problems in the area: treatment of imbalanced data and fully optimizing over continuous variables. We present techniques that produce optimal decision trees over a variety of objectives including F-score, AUC, and partial area under the ROC convex hull. We also introduce a scalable algorithm that produces provably optimal results in the presence of continuous variables and speeds up decision tree construction by several orders of magnitude relative to the state-of-the art.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑