arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1412.5949cs.LG
收藏

大规模分布式距离度量学习

Large Scale Distributed Distance Metric Learning

Carnegie Mellon University(卡内基梅隆大学)

Pengtao Xie, Eric Xing

摘要:

在具有高特征维度的大规模机器学习和数据采掘问题中,数据点之间的欧几里德距离可能是无信息的,并且通常期望距离度量学习(DML)来学习适当的相似性度量(使用诸如相似或不相似的示例数据对之类的边信息)。然而,现代大数据中的高维度和大量成对约束可能导致Xing等人(2002)中的原始DML公式和后来的扩展的计算成本过高。在本文中,我们提出了一个分布式的DML算法,并在一个参数服务器架构的大规模实现。我们的方法建立在Xing et al.(2002)的可并行化重新表述和异步随机梯度下降优化过程的基础上。据我们所知,这是DML的第一个分布式解决方案,我们表明,在一个具有256个CPU内核的系统上,我们的程序能够在15小时内完成一个包含100万个数据点、22000个特征和2亿个标记数据对的数据集上的DML任务;学习的度量在正确测量距离方面表现出很大的有效性。

英文摘要:

In large scale machine learning and data mining problems with high feature dimensionality, the Euclidean distance between data points can be uninformative, and Distance Metric Learning (DML) is often desired to learn a proper similarity measure (using side information such as example data pairs being similar or dissimilar). However, high dimensionality and large volume of pairwise constraints in modern big data can lead to prohibitive computational cost for both the original DML formulation in Xing et al. (2002) and later extensions. In this paper, we present a distributed algorithm for DML, and a large-scale implementation on a parameter server architecture. Our approach builds on a parallelizable reformulation of Xing et al. (2002), and an asynchronous stochastic gradient descent optimization procedure. To our knowledge, this is the first distributed solution to DML, and we show that, on a system with 256 CPU cores, our program is able to complete a DML task on a dataset with 1 million data points, 22-thousand features, and 200 million labeled data pairs, in 15 hours; and the learned metric shows great effectiveness in properly measuring distances.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑