RankMap：一个基于平台的密集数据集分布式学习框架
RankMap: A Platform-Aware Framework for Distributed Learning from Dense Datasets
摘要：
本文介绍了RankMap，一个平台感知的端到端框架，用于高效执行大量密集数据集的广泛的迭代学习算法。我们的框架利用数据结构分解成一个较低的秩子空间的合奏。因子分解创建稀疏的低维表示的数据，这是一个属性，利用设计有效的映射和调度的迭代学习算法的分布式计算机器。我们提供了两个API，一个基于矩阵，一个基于图形，这有利于自动采用的框架执行几个当代的学习应用程序。为了证明RankMap的实用性，我们在各种真实世界的数据集上解决了稀疏恢复和幂迭代问题，这些数据集具有多达18亿个非零值。我们的评估是在使用多达244个内核的Amazon EC2和IBM iDataBase服务器上执行的。结果表明，与最好的报告先前的工作相比，在内存使用，执行速度和带宽方面有两个数量级的改进，同时达到相同的学习精度水平。
英文摘要：
This paper introduces RankMap, a platform-aware end-to-end framework for efficient execution of a broad class of iterative learning algorithms for massive and dense datasets. Our framework exploits data structure to factorize it into an ensemble of lower rank subspaces. The factorization creates sparse low-dimensional representations of the data, a property which is leveraged to devise effective mapping and scheduling of iterative learning algorithms on the distributed computing machines. We provide two APIs, one matrix-based and one graph-based, which facilitate automated adoption of the framework for performing several contemporary learning applications. To demonstrate the utility of RankMap, we solve sparse recovery and power iteration problems on various real-world datasets with up to 1.8 billion non-zeros. Our evaluations are performed on Amazon EC2 and IBM iDataPlex servers using up to 244 cores. The results demonstrate up to two orders of magnitude improvements in memory usage, execution speed, and bandwidth compared with the best reported prior work, while achieving the same level of learning accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
