倾斜随机决策树与森林的统计优势

高维随机镶嵌森林的极小极大收敛速率

Minimax Rates for High-Dimensional Random Tessellation Forests

本文证明一大类采用一般分裂方向的随机森林（包括 STIT forests 和 Poisson 超平面镶嵌导出的森林）在任意维度上都能达到极小极大最优收敛速率，首次表明斜向分裂随机森林变体具有极小极大最优性。

Estimating the Algorithmic Variance of Randomized Ensembles via the Bootstrap

Although the methods of bagging and random forests are some of the most widely used prediction methods, relatively little is known about their algorithmic convergence. In particular, there are not many theoretical guarantees for deciding when an ensemble is "large enough" --- so that its accuracy is close to that of an ideal infinite ensemble. Due to the fact that bagging and random forests are randomized algorithms, the choice of ensemble size is closely related to the notion of "algorithmic variance" (i.e. the variance of prediction error due only to the training algorithm). In the present w…

Universal consistency and minimax rates for online Mondrian Forests

We establish the consistency of an algorithm of Mondrian Forests, a randomized classification algorithm that can be implemented online. First, we amend the original Mondrian Forest algorithm, that considers a fixed lifetime parameter. Indeed, the fact that this parameter is fixed hinders the statistical consistency of the original procedure. Our modified Mondrian Forest algorithm grows trees with increasing lifetime parameters $λ_n$, and uses an alternative updating rule, allowing to work also in an online fashion. Second, we provide a theoretical analysis establishing simple conditions for co…

斜决策树及其提升与随机森林的一致性

Consistency of Oblique Decision Tree and its Boosting and Random Forest

本文证明了斜决策树（ODT）对$L^2$可积的一般回归函数具有一致性，还证明了基于ODT的随机森林（ODRF）的一致性，并提出一种基于正交匹配追踪的GBT集成且验证其一致性，实验显示相关方法相比RF等有显著改进。

Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting

When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose of the current paper is to develop a bootstrap method for solving this problem in the context of regression --- which complements our companion paper in the context of classification (Lopes 2019). In contrast to the classification setting, the current paper shows that theoretica…

Estimating a sharp convergence bound for randomized ensembles

When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship between prediction error and ensemble size is unknown in practice. In the standard case when classifiers are aggregated by majority vote, the present work offers a way to quantify this convergence in terms of "algorithmic variance," i.e. the variance of prediction error due only to the randomized training algorithm. Specifically, we study a theoretical upper …