arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.08884cs.LG
收藏

亿样本半监督学习的低秩标签传播算法

Low-rank Label Propagation for Semi-supervised Learning with 100 Millions Samples

University of Minnesota Twin Cities(明尼苏达大学双城分校)

Raphael Petegrosso, Wei Zhang, Zhuliu Li, Yousef Saad, Rui Kuang

摘要:

半监督学习的成功关键取决于对大量未标记数据的可扩展性,这些数据是捕获底层流形结构以进行更好分类所需的。由于在大多数种类的输入数据中计算训练数据之间的成对相似性是非常昂贵的,因此目前,没有通用的现成的半监督学习方法/工具可用于学习数千万或更多的数据点。本文采用了两种低秩标签传播算法GLNP(Global Linear Neighborhood Propagation)和核Nyström Approximation的思想,实现了这两种算法的并行化版本,并使用Nesterov的加速投影梯度下降算法对大数据标签传播(BigLP)进行加速。并行算法在5个实际数据集上进行了测试,数据集的大小从7000到10,000,000,以及100,000,000个样本的模拟数据集。在实验中,该实现可以扩展到具有100,000,000个样本和数百个特征的数据集,并且当只有很小比例的数据被标记时,该算法也显着提高了预测精度。结果表明,BigLP实现是高度可扩展的大数据和有效地利用未标记的数据进行半监督学习。

英文摘要:

The success of semi-supervised learning crucially relies on the scalability to a huge amount of unlabelled data that are needed to capture the underlying manifold structure for better classification. Since computing the pairwise similarity between the training data is prohibitively expensive in most kinds of input data, currently, there is no general ready-to-use semi-supervised learning method/tool available for learning with tens of millions or more data points. In this paper, we adopted the idea of two low-rank label propagation algorithms, GLNP (Global Linear Neighborhood Propagation) and Kernel Nyström Approximation, and implemented the parallelized version of the two algorithms accelerated with Nesterov's accelerated projected gradient descent for Big-data Label Propagation (BigLP). The parallel algorithms are tested on five real datasets ranging from 7000 to 10,000,000 in size and a simulation dataset of 100,000,000 samples. In the experiments, the implementation can scale up to datasets with 100,000,000 samples and hundreds of features and the algorithms also significantly improved the prediction accuracy when only a very small percentage of the data is labeled. The results demonstrate that the BigLP implementation is highly scalable to big data and effective in utilizing the unlabeled data for semi-supervised learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑