arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.00819cs.LGcs.DSstat.ML
收藏

即使在平滑设置中,决策树算法也可能失败

Decision tree heuristics can fail, even in the smoothed setting

Stanford(斯坦福大学);MIT(麻省理工学院)

Guy Blanc, Jane Lange, Mingda Qiao, Li-Yang Tan

摘要:

贪婪决策树学习算法是机器学习实践的支柱,但其经验成功的理论依据仍然难以捉摸。事实上,人们早就知道,有一些简单的目标功能,他们失败严重(卡恩斯和曼苏尔,STOC 1996)。Brutzkus,Daniely和Malach(COLT 2020)最近的工作认为平滑分析模型是解决这种脱节的可能途径。在平滑设置中,对于$k$ -juntas的目标$f$,他们表明这些算法成功地学习了具有深度$k$决策树假设的$f$。他们强调,对于深度$k$决策树的目标,同样的保证更普遍。我们提供了一个反例,这个猜想:我们构建的目标是深度$k$决策树,并表明,即使在平滑设置,这些算法建立树的深度$2^{Ω(k)}$之前,实现高精度。我们还表明,Brutzkus等人的保证。不能扩展到不可知的设置:有非常接近$k$ -juntas的目标,这些算法建立树的深度$2^{Ω(k)}$之前实现高精度。

英文摘要:

Greedy decision tree learning heuristics are mainstays of machine learning practice, but theoretical justification for their empirical success remains elusive. In fact, it has long been known that there are simple target functions for which they fail badly (Kearns and Mansour, STOC 1996). Recent work of Brutzkus, Daniely, and Malach (COLT 2020) considered the smoothed analysis model as a possible avenue towards resolving this disconnect. Within the smoothed setting and for targets $f$ that are $k$-juntas, they showed that these heuristics successfully learn $f$ with depth-$k$ decision tree hypotheses. They conjectured that the same guarantee holds more generally for targets that are depth-$k$ decision trees. We provide a counterexample to this conjecture: we construct targets that are depth-$k$ decision trees and show that even in the smoothed setting, these heuristics build trees of depth $2^{Ω(k)}$ before achieving high accuracy. We also show that the guarantees of Brutzkus et al. cannot extend to the agnostic setting: there are targets that are very close to $k$-juntas, for which these heuristics build trees of depth $2^{Ω(k)}$ before achieving high accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑