决策树的自顶向下归纳：严格保证和固有局限性
Top-down induction of decision trees: rigorous guarantees and inherent limitations
摘要：
考虑以下为函数$f : \{0,1\}^n \to \{\pm 1\}$构建决策树的启发式方法。将$f$的最有影响力的变量$x_i$放在根上，并分别递归左、右子树上的子函数$f_{x_i=0}$和$f_{x_i=1}$;一旦树是$f$的$\varepsilon$ -近似，就终止。我们分析这个启发式的质量，获得接近匹配的上限和下限：$\circ$上限：对于每个$f$与决策树大小$s$和每个$\varepsilon \in (0,\frac1{2})$，这个启发式建立一个决策树的大小最多$s^{O(\log(s/\varepsilon)\log(1/\varepsilon))}$。$\circ$下限：对于每个$\varepsilon \in (0,\frac1{2})$和$s \le 2^{\tilde{O}(\sqrt{n})}$，存在具有决策树大小$s$的$f$，使得该试探法构建大小为$s^{\tildeΩ(\log s)}$的决策树。我们还得到了单调函数$s^{O(\sqrt{\log s}/\varepsilon)}$和$s^{\tildeΩ(\sqrt[4]{\log s } )}$的上下界。这个下界反驳了Fiat and Pechyony（2004）和Lee（2009）的推论。我们的上界产生新的算法正确学习决策树下的均匀分布。我们表明，这些算法-我们的下界揭示了这些逻辑学的局限性。最后，我们重新审视了埃塞尔福赫特和豪斯勒的经典作品。我们将其扩展到第一个均匀分布适当的学习算法，实现多项式样本和内存复杂度，同时匹配其最先进的准多项式运行时。
英文摘要：
Consider the following heuristic for building a decision tree for a function $f : \{0,1\}^n \to \{\pm 1\}$. Place the most influential variable $x_i$ of $f$ at the root, and recurse on the subfunctions $f_{x_i=0}$ and $f_{x_i=1}$ on the left and right subtrees respectively; terminate once the tree is an $\varepsilon$-approximation of $f$. We analyze the quality of this heuristic, obtaining near-matching upper and lower bounds: $\circ$ Upper bound: For every $f$ with decision tree size $s$ and every $\varepsilon \in (0,\frac1{2})$, this heuristic builds a decision tree of size at most $s^{O(\log(s/\varepsilon)\log(1/\varepsilon))}$. $\circ$ Lower bound: For every $\varepsilon \in (0,\frac1{2})$ and $s \le 2^{\tilde{O}(\sqrt{n})}$, there is an $f$ with decision tree size $s$ such that this heuristic builds a decision tree of size $s^{\tildeΩ(\log s)}$. We also obtain upper and lower bounds for monotone functions: $s^{O(\sqrt{\log s}/\varepsilon)}$ and $s^{\tildeΩ(\sqrt[4]{\log s } )}$ respectively. The lower bound disproves conjectures of Fiat and Pechyony (2004) and Lee (2009). Our upper bounds yield new algorithms for properly learning decision trees under the uniform distribution. We show that these algorithms---which are motivated by widely employed and empirically successful top-down decision tree learning heuristics such as ID3, C4.5, and CART---achieve provable guarantees that compare favorably with those of the current fastest algorithm (Ehrenfeucht and Haussler, 1989). Our lower bounds shed new light on the limitations of these heuristics. Finally, we revisit the classic work of Ehrenfeucht and Haussler. We extend it to give the first uniform-distribution proper learning algorithm that achieves polynomial sample and memory complexity, while matching its state-of-the-art quasipolynomial runtime.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
