arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03386cs.LGcs.AI

一种基于神经网络架构的广义决策树集成:分布式梯度提升森林(DGBF)

A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)

Paradigma Digital S.L.(Paradigma Digital有限公司);Universidad de León(莱昂大学)

Ángel Delgado-Panadero, José Alberto Benítez-Andrades, María Teresa García-Ordás

AI总结:

提出分布式梯度提升森林(DGBF),融合装袋与提升的数学表述,构建图结构树集成,无需反向传播即可实现分布式表示学习,在9个数据集中7个上优于随机森林和梯度提升。

摘要:

树集成算法如随机森林和梯度提升目前是建模离散或表格数据的主要方法,然而,它们无法像神经网络那样凭借其多层结构从原始数据中进行层次化表示学习,而这是深度学习问题和建模非结构化数据的关键特征。这一局限性源于树算法因其数学性质无法通过反向传播进行训练。然而,在本工作中,我们证明了装袋(bagging)和提升(boosting)的数学表述可以结合起来,定义一种图结构树集成算法,其在树之间自然地具有分布式表示学习过程(无需使用反向传播)。我们将这种新方法称为分布式梯度提升森林(DGBF),并证明随机森林和梯度提升都可以表示为DGBF的特定图架构。最后,我们看到分布式学习在9个数据集中的7个上优于随机森林和梯度提升。

英文摘要:

Tree ensemble algorithms as RandomForest and GradientBoosting are currently the dominant methods for modeling discrete or tabular data, however, they are unable to perform a hierarchical representation learning from raw data as NeuralNetworks does thanks to its multi-layered structure, which is a key feature for DeepLearning problems and modeling unstructured data. This limitation is due to the fact that tree algorithms can not be trained with back-propagation because of their mathematical nature. However, in this work, we demonstrate that the mathematical formulation of bagging and boosting can be combined together to define a graph-structured-tree-ensemble algorithm with a distributed representation learning process between trees naturally (without using back-propagation). We call this novel approach Distributed Gradient Boosting Forest (DGBF) and we demonstrate that both RandomForest and GradientBoosting can be expressed as particular graph architectures of DGBT. Finally, we see that the distributed learning outperforms both RandomForest and GradientBoosting in 7 out of 9 datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑