发表机构
Université Paris-Saclay; UVSQ; Inserm; CESP; Université Paris Cité; ENS Paris-Saclay; CNRS; SSA; Centre Borelli; École du Val-de-Grâce; Service de Santé des Armées（巴黎萨克雷大学; 凡尔赛圣康丁伊夫林大学; 法国国家健康与医学研究院; 健康与医学研究中心; 巴黎西岱大学; 巴黎萨克雷高等师范学院; 法国国家科学研究中心; 法国军队卫生服务局; 博雷利中心; 瓦尔-德-格拉斯学校; 法国军队卫生服务局）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对多输出预测中条件分布估计的难题，提出并行梯度提升算法，通过共同下降方向减少计算量，在多变量回归设置中性能优于现有库，所得估计器在高维及复杂协变量情况下表现出色。
AI 中文摘要
提升是标准分类和回归任务中最成功的学习技术之一。其扩展到多输出预测问题近年来应用越来越多，比如预测整个条件分布。用经典提升实现处理此类问题计算量很大，因为每次迭代通常为每个目标训练一个基模型。本文研究了梯度提升算法的一种修改——并行梯度提升，核心是对所有训练观测使用共同下降方向，每次迭代只需一个基模型，性能大幅提升。我们建立了算法收敛的充分条件，通过多变量回归设置介绍其实际应用。结果表明，在多变量回归设置中，它能提供与XGBoost等库类似质量的预测，且快几个数量级。此外，评估了所得条件分布估计器的属性，实证表明其优于其他非参数和半参数估计器，尤其在高维设置以及存在混合和/或缺失协变量的情况下。
英文摘要
Boosting is one of the most successful learning techniques for standard classification and regression tasks. Its extension to multi-output prediction problems has found an increasing number of applications in recent years. Among them is the prediction of entire conditional distributions rather than single functionals, which can often be framed as a multi-output regression problem, for example multiple quantile regression. Addressing such problems with classical implementations of boosting is computationally challenging, because usually one base model is trained for each target at every iteration. More efficient variants of boosting have been proposed to speed up training, but they tend to be tied to specific loss functions and classes of base learners, usually decision trees. In this work, we study a modification of the gradient boosting algorithm, which we call parallel gradient boosting, designed to circumvent all these limitations. The core idea is to use a common descent direction for all training observations. By doing so, only one base model is needed at each iteration, regardless of the number of targets, which allows for considerable performance gains. We establish sufficient conditions for the convergence of the algorithm, whose practical use is introduced via the multiple quantile regression setting. We show that in such a setting, it provides predictions of similar quality to state-of-the-art boosting libraries such as XGBoost, while being faster by several orders of magnitude. Then, we evaluate the properties of the resulting conditional distribution estimator, which is shown empirically to outperform other nonparametric and semiparametric estimators, especially in high-dimensional settings and in the presence of mixed and/or missing covariates.
