arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.00699cs.DCcs.PF
收藏

数据侏儒:全面了解大数据和AI工作负载的视角

Data Dwarfs: A Lens Towards Fully Understanding Big Data and AI Workloads

Wanling Gao, Jianfeng Zhan, Lei Wang, Chunjie Luo, Daoyi Zheng, Fei Tang, Biwei Xie, Chen Zheng, Qiang Yang

摘要:

大数据和人工智能工作负载的复杂性和多样性使得理解它们变得困难和具有挑战性。本文提出了一种表征大数据和AI工作负载的新方法。我们将每个大数据和人工智能工作负载视为对不同初始或中间数据输入执行的一类或多类计算单元的管道。每一类计算单元都捕获了共同的需求,同时又与单独的实现合理地分离,因此我们称之为数据侏儒。在各种各样的大数据和人工智能工作负载中,我们第一次确定了八个占用大部分运行时间的数据侏儒,包括矩阵,采样,逻辑,变换,集合,图形,排序和统计。我们在不同的软件栈上实现了八个数据侏儒,作为开源大数据和人工智能基准测试套件的微观基准,并从数据大小,类型,来源和模式的角度对这些数据侏儒进行全面表征,作为全面理解大数据和人工智能工作负载的镜头。

英文摘要:

The complexity and diversity of big data and AI workloads make understanding them difficult and challenging. This paper proposes a new approach to characterizing big data and AI workloads. We consider each big data and AI workload as a pipeline of one or more classes of unit of computations performed on different initial or intermediate data inputs. Each class of unit of computation captures the common requirements while being reasonably divorced from individual implementations, and hence we call it a data dwarf. For the first time, among a wide variety of big data and AI workloads, we identify eight data dwarfs that takes up most of run time, including Matrix, Sampling, Logic, Transform, Set, Graph, Sort and Statistic. We implement the eight data dwarfs on different software stacks as the micro benchmarks of an open-source big data and AI benchmark suite, and perform comprehensive characterization of those data dwarfs from perspective of data sizes, types, sources, and patterns as a lens towards fully understanding big data and AI workloads.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑