We consider $L_2$Boosting, a special case of Friedman's generic boosting algorithm applied to linear regression under $L_2$-loss. We study $L_2$Boosting for an arbitrary regularization parameter and derive an exact closed form expression for the number of steps taken along a fixed coordinate direction. This relationship is used to describe $L_2$Boosting's solution path, to describe new tools for studying its path, and to characterize some of the algorithm's unique properties, including active set cycling, a property where the algorithm spends lengthy periods of time cycling between the same coordinates when the regularization parameter is arbitrarily small. Our fixed descent analysis also reveals a repressible condition that limits the effectiveness of $L_2$Boosting in correlated problems by preventing desirable variables from entering the solution path. As a simple remedy, a data augmentation method similar to that used for the elastic net is used to introduce $L_2$-penalization and is shown, in combination with decorrelation, to reverse the repressible condition and circumvents $L_2$Boosting's deficiencies in correlated problems. In itself, this presents a new explanation for why the elastic net is successful in correlated problems and why methods like LAR and lasso can perform poorly in such settings.
OT-FairBoost: Optimal Transport-Guided Gradient Boosting for Fairness Regularization on Tabular Data
Artefact Research Center(Artefact 研究中心);Institut de Mathématique de Toulouse, CNRS(图卢兹数学研究所,法国国家科学研究中心);Laboratoire de Probabilités, Statistique et Modélisation Sorbonne Université Université Paris Cité, CNRS(索邦大学巴黎西岱大学概率、统计与建模实验室,法国国家科学研究中心);Université de Toulouse, ANITI(图卢兹大学,ANITI);INRIA Regalia, Bordeaux(法国国家信息与自动化研究所 Regalia 团队,波尔多)
MIBoost: A gradient boosting algorithm for variable selection after multiple imputation
Institute of Medical Biostatistics, Epidemiology and Informatics (IMBEI), University Medical Center, Johannes Gutenberg University Mainz(医学生物统计学、流行病学与信息学研究所(IMBEI),大学医学中心,美因茨约翰内斯·古腾堡大学)
Statistical Inference for Gradient Boosting Regression
School of Economics, Fudan University(复旦大学经济学院);Department of Statistics and Data Science, The Wharton School, University of Pennsylvania(宾夕法尼亚大学沃顿商学院统计与数据科学系)