减少错误修剪的分析
An Analysis of Reduced Error Pruning
摘要:
决策树的自顶向下归纳已经被观察到遭受修剪阶段的功能不足。特别是,它是已知的,所得到的树的大小与样本大小线性增长,即使树的准确性没有提高。减少错误修剪是一种算法,已被用作试图解释决策树学习问题的代表技术。在本文中,我们提出了分析减少错误修剪在三个不同的设置。首先,我们研究了该方法的基本算法性质,独立于输入决策树的性质和剪枝实例。然后,我们研究的情况下,直观地应导致考虑中的子树被替换为叶节点,其中的类标签和属性值的修剪的例子是相互独立的。这一分析是在两种不同的假设下进行的。一般的分析表明,一个节点拟合纯噪声的修剪概率是由一个函数,随着树的大小呈指数下降的范围。在一个具体的分析中,我们假设样本均匀分布到树中。这个假设让我们近似修剪的子树的数量,因为他们没有收到任何修剪的例子。本文阐明了减少错误修剪算法的不同变体,带来了新的见解,其算法特性,分析算法与以前相比,施加较少的假设,并包括以前被忽视的空子树的分析。
英文摘要:
Top-down induction of decision trees has been observed to suffer from the inadequate functioning of the pruning phase. In particular, it is known that the size of the resulting tree grows linearly with the sample size, even though the accuracy of the tree does not improve. Reduced Error Pruning is an algorithm that has been used as a representative technique in attempts to explain the problems of decision tree learning. In this paper we present analyses of Reduced Error Pruning in three different settings. First we study the basic algorithmic properties of the method, properties that hold independent of the input decision tree and pruning examples. Then we examine a situation that intuitively should lead to the subtree under consideration to be replaced by a leaf node, one in which the class label and attribute values of the pruning examples are independent of each other. This analysis is conducted under two different assumptions. The general analysis shows that the pruning probability of a node fitting pure noise is bounded by a function that decreases exponentially as the size of the tree grows. In a specific analysis we assume that the examples are distributed uniformly to the tree. This assumption lets us approximate the number of subtrees that are pruned because they do not receive any pruning examples. This paper clarifies the different variants of the Reduced Error Pruning algorithm, brings new insight to its algorithmic properties, analyses the algorithm with less imposed assumptions than before, and includes the previously overlooked empty subtrees to the analysis.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。