We use the output of a random forest to define a family of local smoothers with spatially adaptive bandwidth matrices. The smoother inherits the flexibility of the original forest but, since it is a simple, linear smoother, it is very interpretable and it can be used for tasks that would be intractable for the original forest. This includes bias correction, confidence intervals, assessing variable importance and methods for exploring the structure of the forest. We illustrate the method on some synthetic examples and on data related to Covid-19.
This paper is a comment on the survey paper by Biau and Scornet (2016) about random forests. We focus on the problem of quantifying the impact of each ingredient of random forests on their performance. We show that such a quantification is possible for a simple pure forest , leading to conclusions that could apply more generally. Then, we consider "hold-out" random forests, which are a good middle point between "toy" pure forests and Breiman's original random forests.
Guided Random Forest and its application to data approximation
Department of Electrical Engineering, Indian Institute of Technology Delhi（印度理工学院德里分校电子工程系）；Center for Computational Biology, Indraprastha Institute of Information Technology（印度信息科技学院计算生物学中心）；Department of Computer Science and Engineering, Indraprastha Institute of Information Technology（印度信息科技学院计算机科学与工程系）；Infosys Center for Artificial Intelligence, Indraprastha Institute of Information Technology（印度信息科技学院人工智能中心）；Institute of Health and Biomedical Innovation, Queensland University of Technology（昆士兰科技大学健康与生物医学创新研究所）；Department of Mathematics, Indian Institute of Technology Delhi（印度理工学院德里分校数学系）
The random forest algorithm, proposed by L. Breiman in 2001, has been extremely successful as a general-purpose classification and regression method. The approach, which combines several randomized decision trees and aggregates their predictions by averaging, has shown excellent performance in settings where the number of variables is much larger than the number of observations. Moreover, it is versatile enough to be applied to large-scale problems, is easily adapted to various ad-hoc learning tasks, and returns measures of variable importance. The present article reviews the most recent theor…
With inspiration from Random Forests (RF) in the context of classification, a new clustering ensemble method---Cluster Forests (CF) is proposed. Geometrically, CF randomly probes a high-dimensional data cloud to obtain "good local clusterings" and then aggregates via spectral clustering to obtain cluster assignments for the whole dataset. The search for good local clusterings is guided by a cluster quality measure kappa. CF progressively improves each local clustering in a fashion that resembles the tree growth in RF. Empirical studies on several real-world datasets under two different perform…
In this study we investigate the potential for using synthetic aperture radar (SAR) data to provide high resolution defoliation and regrowth mapping of trees in the tundra-forest ecotone. Using aerial photographs, four areas with live forest and four areas with dead trees were identified. Quad-polarimetric SAR data from RADARSAT-2 was collected from the same area, and the complex multilook polarimetric covariance matrix was calculated using a novel extension of guided nonlocal means speckle filtering. The nonlocal approach allows us to preserve the high spatial resolution of single-look comple…
We consider finding a counterfactual explanation for a classification or regression forest, such as a random forest. This requires solving an optimization problem to find the closest input instance to a given instance for which the forest outputs a desired value. Finding an exact solution has a cost that is exponential on the number of leaves in the forest. We propose a simple but very effective approach: we constrain the optimization to only those input space regions defined by the forest that are populated by actual data points. The problem reduces to a form of nearest-neighbor search using …
High-Dimensional Dynamic Covariance Models with Random Forests
School of Statistics and Management, Shanghai University of Finance and Economics（上海财经大学统计与管理学院）；School of Statistics, KLATASDS-MOE, East China Normal University（华东师范大学统计学院）；Departments of Biostatistics, Statistics, Computer Science, and Genetics, The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校生物统计学、统计学、计算机科学和遗传学系）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Infinite-order U-statistics (IOUS) has been used extensively on subbagging ensemble learning algorithms such as random forests to quantify its uncertainty. While normality results of IOUS have been studied extensively, its variance estimation approaches and theoretical properties remain mostly unexplored. Existing approaches mainly utilize the leading term dominance property in the Hoeffding decomposition. However, such a view usually leads to biased estimation when the kernel size is large or the sample size is small. On the other hand, while several unbiased estimators exist in the literatur…
DiNo and RanBu: Lightweight Predictions from Shallow Random Forests
Insper Institute of Education and Research（Insper教育与研究学院）；Department of Statistics（统计系）；Federal University of São Carlos（圣卡洛斯联邦大学）；Institute of Mathematics and Statistics（数学与统计学研究所）；University of São Paulo（圣保罗大学）
In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from $\mathcal{O}(\sqrt\frac{\ln m}{m})$ to $\mathcal{O}(\frac{\ln m}{m})$, while $λ$ - the margin ratio between the margin standard deviation and the margin mean is small enough. This tighter upper bound inspires us to optimize the margin distribution ratio $λ$. Therefore, we design the margin distribution reweighting approach (mdDF) to achieve small ratio $λ$ by boosting the …
Label-Guided Imputation via Forest-Based Proximities for Improved Time Series Classification
1 Department of Statistics, Brigham Young University, Provo, UT, USA Emails；2 Department of Computer Science, Brigham Young University, Provo, UT, USA Email；3 Department of Public Health, Brigham Young University, Provo, UT, USA Email；4 Department of Neurology, Vagelos College of Physicians \& Surgeons, Columbia University, New York, NY, USA Email；5 Department of Neurology, Miller School of Medicine, University of Miami, Miami, FL, USA Email；6 Elmhurst Hospital Center, NYC Health + Hospitals, New York, NY, USA Email；7 Department of Mathematics \& Statistics, Utah State University, Logan, UT, USA Email
