Recursive Bias Estimation and $L_2$ Boosting

Boosting Random Forests to Reduce Bias; One-Step Boosted Forest and its Variance Estimate

In this paper we propose using the principle of boosting to reduce the bias of a random forest prediction in the regression setting. From the original random forest fit we extract the residuals and then fit another random forest to these residuals. We call the sum of these two random forests a \textit{one-step boosted forest}. We show with simulated and real data that the one-step boosted forest has a reduced bias compared to the original random forest. The paper also provides a variance estimate of the one-step boosted forest by an extension of the infinitesimal Jackknife estimator. Using thi…

Recursive bias estimation for multivariate regression smoothers

This paper presents a practical and simple fully nonparametric multivariate smoothing procedure that adapts to the underlying smoothness of the true regression function. Our estimator is easily computed by successive application of existing base smoothers (without the need of selecting an optimal smoothing parameter), such as thin-plate spline or kernel smoothers. The resulting smoother has better out of sample predictive capabilities than the underlying base smoother, or competing structurally constrained models (GAM) for small dimension (3 < d < 8) and moderate sample size (n < 800). Moreove…

Gradient Boosting for Linear Mixed Models

Gradient boosting from the field of statistical learning is widely known as a powerful framework for estimation and selection of predictor effects in various regression models by adapting concepts from classification theory. Current boosting approaches also offer methods accounting for random effects and thus enable prediction of mixed models for longitudinal and clustered data. However, these approaches include several flaws resulting in unbalanced effect selection with falsely induced shrinkage and a low convergence rate on the one hand and biased estimates of the random effects on the other…

基于随机和固定效应的空间面板模型的梯度提升

Gradient Boosting for Spatial Panel Models with Random and Fixed Effects

本文提出基于模型的梯度提升算法，用于估计具有随机和固定效应的空间面板模型，适用于高维和低维设置，通过数据驱动的模型和变量选择提升预测准确性。

Boosting Joint Models for Longitudinal and Time-to-Event Data

Joint Models for longitudinal and time-to-event data have gained a lot of attention in the last few years as they are a helpful technique to approach common a data structure in clinical studies where longitudinal outcomes are recorded alongside event times. Those two processes are often linked and the two outcomes should thus be modeled jointly in order to prevent the potential bias introduced by independent modelling. Commonly, joint models are estimated in likelihood based expectation maximization or Bayesian approaches using frameworks where variable selection is problematic and which do no…

TERRA：一种用于纵向异质性治疗效果估计的Transformer赋能的递归R学习器

TERRA: A Transformer-Enabled Recursive R-learner for Longitudinal Heterogeneous Treatment Effect Estimation

TERRA提出一种Transformer赋能的递归R学习器，通过编码治疗历史和递归残差学习，解决纵向HTE估计中的遗留效应与治疗后偏倚，并在模拟和实际数据中优于基线。