arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1904.10522cs.LGcs.IRstat.ML
收藏

块分布梯度提升树

Block-distributed Gradient Boosted Trees

RISE AI;Amazon Web Services(亚马逊云科技);KTH Royal Institute of Technology(瑞典皇家理工学院)

Theodore Vasiloudis, Hyunsu Cho, Henrik Boström

摘要:

梯度提升树(GBT)算法是生产中最流行的机器学习算法之一,适用于包括点击率(CTR)预测和学习排名在内的任务。为了处理大量的数据集,目前已经提出了许多分布式GBT方法。然而,它们都假设了一个行分布的数据集,只针对数据点的数量而不是特征的数量来解决可伸缩性,并增加了高维数据的通信成本。为了允许跨数据点和特征维度的可扩展性,并降低通信成本,我们提出了块分布式GBT。通过充分利用数据的稀疏性,并将Quickscorer算法应用于块分布环境,提高了通信效率。我们使用具有数百万特征的数据集来评估我们的方法,并证明我们能够在不损失准确性的情况下实现稀疏数据通信成本的多个数量级降低,同时提供更具可扩展性的设计。因此,我们能够减少高维数据的训练时间,并允许更具成本效益的横向扩展,而无需昂贵的网络通信。

英文摘要:

The Gradient Boosted Tree (GBT) algorithm is one of the most popular machine learning algorithms used in production, for tasks that include Click-Through Rate (CTR) prediction and learning-to-rank. To deal with the massive datasets available today, many distributed GBT methods have been proposed. However, they all assume a row-distributed dataset, addressing scalability only with respect to the number of data points and not the number of features, and increasing communication cost for high-dimensional data. In order to allow for scalability across both the data point and feature dimensions, and reduce communication cost, we propose block-distributed GBTs. We achieve communication efficiency by making full use of the data sparsity and adapting the Quickscorer algorithm to the block-distributed setting. We evaluate our approach using datasets with millions of features, and demonstrate that we are able to achieve multiple orders of magnitude reduction in communication cost for sparse data, with no loss in accuracy, while providing a more scalable design. As a result, we are able to reduce the training time for high-dimensional data, and allow more cost-effective scale-out without the need for expensive network communication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑