arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1704.07657cs.LG
收藏

决策流:培育深层决策树

Decision Stream: Cultivating Deep Decision Trees

Russian Research Center of Huawei Technologies(华为技术俄罗斯研究中心);ETH Zurich(苏黎世联邦理工学院)

Dmitry Ignatov, Andrey Ignatov

摘要:

决策树的各种修改由于其高效率和可解释性在过去的几年中得到了广泛的应用。基于相关特征选择的决策树节点划分是决策树学习的关键步骤,同时也是其主要缺点:递归节点划分导致叶节点数据量几何减少,导致模型复杂度过高和数据过拟合。在本文中,我们提出了一种新的架构-决策流,-旨在克服这个问题。在学习过程中,我们提出了合并节点从不同的分支,而不是建立一个树结构的基础上,他们的相似性,估计与两个样本的测试统计,这导致生成的决策规则,可以包括数百个级别的深度有向无环图。为了评估所提出的解决方案,我们在几个常见的机器学习问题上进行了测试-信用评分,twitter情感分析,飞机飞行控制,MNIST和CIFAR图像分类,合成数据分类和回归。我们的实验结果表明,所提出的方法在回归和分类任务上都显着优于标准的决策树学习方法,预测误差降低高达35%。

英文摘要:

Various modifications of decision trees have been extensively used during the past years due to their high efficiency and interpretability. Tree node splitting based on relevant feature selection is a key step of decision tree learning, at the same time being their major shortcoming: the recursive nodes partitioning leads to geometric reduction of data quantity in the leaf nodes, which causes an excessive model complexity and data overfitting. In this paper, we present a novel architecture - a Decision Stream, - aimed to overcome this problem. Instead of building a tree structure during the learning process, we propose merging nodes from different branches based on their similarity that is estimated with two-sample test statistics, which leads to generation of a deep directed acyclic graph of decision rules that can consist of hundreds of levels. To evaluate the proposed solution, we test it on several common machine learning problems - credit scoring, twitter sentiment analysis, aircraft flight control, MNIST and CIFAR image classification, synthetic data classification and regression. Our experimental results reveal that the proposed approach significantly outperforms the standard decision tree learning methods on both regression and classification tasks, yielding a prediction error decrease up to 35%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑