arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.01341cs.DC
收藏

AccurateML:基于信息聚合的近似处理,用于MapReduce上的快速准确机器学习

AccurateML: Information-aggregation-based Approximate Processing for Fast and Accurate Machine Learning on MapReduce

Rui Han, Fan Zhang, Zhentao Wang

摘要:

处理海量数据集的需求不断增长,推动了在MapReduce上运行机器学习应用程序的不可阻挡的趋势。当处理大量输入数据时,产生快速且足够准确的近似结果通常比缓慢的精确结果具有更大的价值。现有技术通过处理部分输入数据来产生近似结果,因此当使用短作业执行时间时会导致大的准确度损失,因为所有跳过的输入数据潜在地有助于结果准确度。我们通过提出AccurateML来解决这个限制,AccurateML聚合每个映射任务中的输入数据的信息,以创建小的聚合数据点。这些聚合点使所有地图任务能够快速生成初始输出,以节省计算时间并减少输出的大小,从而减少通信时间。我们的方法进一步识别输入数据中与结果精度最相关的部分,从而首先使用这些部分来改进所产生的输出以最小化精度损失。我们使用真实的机器学习应用程序和数据集评估了AccurateML。结果表明:(i)与精确结果相比,它减少了30倍的执行时间,精度损失很小;(ii)当使用相同的执行时间时,与现有的近似处理技术相比,它实现了精度损失的2.71倍减少。

英文摘要:

The growing demands of processing massive datasets have promoted irresistible trends of running machine learning applications on MapReduce. When processing large input data, it is often of greater values to produce fast and accurate enough approximate results than slow exact results. Existing techniques produce approximate results by processing parts of the input data, thus incurring large accuracy losses when using short job execution times, because all the skipped input data potentially contributes to result accuracy. We address this limitation by proposing AccurateML that aggregates information of input data in each map task to create small aggregated data points. These aggregated points enable all map tasks producing initial outputs quickly to save computation times and decrease the outputs' size to reduce communication times. Our approach further identifies the parts of input data most related to result accuracy, thus first using these parts to improve the produced outputs to minimize accuracy losses. We evaluated AccurateML using real machine learning applications and datasets. The results show: (i) it reduces execution times by 30 times with small accuracy losses compared to exact results; (ii) when using the same execution times, it achieves 2.71 times reductions in accuracy losses compared to existing approximate processing techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑