arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.4620cs.LGstat.ML
收藏

改进的决策树归纳信息增益估计

Improved Information Gain Estimates for Decision Tree Induction

Microsoft Research Cambridge(微软研究院剑桥分部)

Sebastian Nowozin

摘要:

分类树和回归树的集合仍然是流行的机器学习方法,因为它们定义了灵活的非参数模型,这些模型在训练和测试期间预测良好,并且计算效率高。在决策树的归纳过程中,我们的目标是找到关于预测目标的最大信息的谓词。为了选择好的谓词,大多数方法估计信息理论的评分函数,信息增益,分类和回归问题。我们指出,共同的估计程序是有偏见的,并表明,通过取代它们与改进的离散和微分熵的估计,我们可以得到更好的决策树。实际上,我们的修改产生了改进的预测性能,并且易于在任何决策树代码中实现。

英文摘要:

Ensembles of classification and regression trees remain popular machine learning methods because they define flexible non-parametric models that predict well and are computationally efficient both during training and testing. During induction of decision trees one aims to find predicates that are maximally informative about the prediction target. To select good predicates most approaches estimate an information-theoretic scoring function, the information gain, both for classification and regression problems. We point out that the common estimation procedures are biased and show that by replacing them with improved estimators of the discrete and the differential entropy we can obtain better decision trees. In effect our modifications yield improved predictive performance and are simple to implement in any decision tree code.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑