数据主题:全面理解大数据和AI工作负载的视角
Data Motifs: A Lens Towards Fully Understanding Big Data and AI Workloads
摘要:
大数据和人工智能工作负载的复杂性和多样性使得理解它们变得困难和具有挑战性。本文提出了一种新的方法来建模和表征大数据和人工智能工作负载。我们将每个大数据和人工智能工作负载视为对不同初始或中间数据输入执行的一类或多类计算单元的管道。每一类计算单元都捕获了共同的需求,同时又与单独的实现合理地分离,因此我们称之为数据基元。在各种各样的大数据和人工智能工作负载中,我们第一次确定了八个数据基元,它们占用了这些工作负载的大部分运行时间,包括矩阵,采样,逻辑,变换,集合,图形,排序和统计。我们在不同的软件栈上实现了八个数据基元,作为开源大数据和人工智能基准测试套件BigDataBench 4. 0(可从http://prof.ict.ac.cn/BigDataBench公开获取)的微基准测试,并从数据大小,类型,来源和模式的角度对这些数据基元进行全面表征,作为全面理解大数据和人工智能工作负载的镜头。我们相信这八个数据基元不仅是大数据和人工智能基准测试的有前途的抽象和工具,而且也是特定领域的硬件和软件协同设计。
英文摘要:
The complexity and diversity of big data and AI workloads make understanding them difficult and challenging. This paper proposes a new approach to modelling and characterizing big data and AI workloads. We consider each big data and AI workload as a pipeline of one or more classes of units of computation performed on different initial or intermediate data inputs. Each class of unit of computation captures the common requirements while being reasonably divorced from individual implementations, and hence we call it a data motif. For the first time, among a wide variety of big data and AI workloads, we identify eight data motifs that take up most of the run time of those workloads, including Matrix, Sampling, Logic, Transform, Set, Graph, Sort and Statistic. We implement the eight data motifs on different software stacks as the micro benchmarks of an open-source big data and AI benchmark suite ---BigDataBench 4.0 (publicly available from http://prof.ict.ac.cn/BigDataBench), and perform comprehensive characterization of those data motifs from perspective of data sizes, types, sources, and patterns as a lens towards fully understanding big data and AI workloads. We believe the eight data motifs are promising abstractions and tools for not only big data and AI benchmarking, but also domain-specific hardware and software co-design.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。