AOSO-LogitBoost:多类问题的自适应一对一LogitBoost
AOSO-LogitBoost: Adaptive One-Vs-One LogitBoost for Multi-Class Problem
摘要:
本文提出了使用多类LogitBoost进行分类时对模型学习的改进。受统计视图的启发,LogitBoost可以被视为加法树回归。这种设置中的两个重要因素是:1)由于和零约束而耦合的分类器输出,以及2)在计算树节点分裂增益和节点值拟合时出现的密集Hessian矩阵。通常,这种设置对于易于处理的模型学习算法来说太复杂。但是,过于激进的设置简化可能会导致性能下降。例如,ABC-LogitBoost优于原始LogitBoost,因为后者更仔细地处理了上述两个因素。在本文中,我们提出了解决LogitBoost设置的两个主要困难的技术:1)我们采用了一个矢量树(即每个节点值都是矢量),它强制执行和为零的约束,以及2)我们使用了一个自适应块坐标下降,在计算树分裂增益和节点值时利用了稠密Hessian。与原始和ABC-LogitBoost实现相比,在一系列公共数据集上观察到更高的分类精度和更快的收敛速度。
英文摘要:
This paper presents an improvement to model learning when using multi-class LogitBoost for classification. Motivated by the statistical view, LogitBoost can be seen as additive tree regression. Two important factors in this setting are: 1) coupled classifier output due to a sum-to-zero constraint, and 2) the dense Hessian matrices that arise when computing tree node split gain and node value fittings. In general, this setting is too complicated for a tractable model learning algorithm. However, too aggressive simplification of the setting may lead to degraded performance. For example, the original LogitBoost is outperformed by ABC-LogitBoost due to the latter's more careful treatment of the above two factors. In this paper we propose techniques to address the two main difficulties of the LogitBoost setting: 1) we adopt a vector tree (i.e. each node value is vector) that enforces a sum-to-zero constraint, and 2) we use an adaptive block coordinate descent that exploits the dense Hessian when computing tree split gain and node values. Higher classification accuracy and faster convergence rates are observed for a range of public data sets when compared to both the original and the ABC-LogitBoost implementations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。