面向高性能数据科学和数据工程的HPTMT并行运算器
HPTMT Parallel Operators for High Performance Data Science & Data Engineering
摘要:
数据密集型应用程序在所有科学学科中变得越来越普遍。它们由一组丰富的子领域组成,如数据工程、深度学习和机器学习。这些应用程序都是围绕有效的数据抽象和运营商,适合不同领域的应用程序。在该领域中经常缺乏对数据结构和运算符的明确定义,这导致了其他实现不能很好地协同工作。我们最近提出的HPTMT架构确定了一组数据结构、操作符和执行模型,用于创建将数据工程和数据科学的所有方面有效地联系在一起的丰富数据应用程序。本文使用一个端到端应用程序来阐述和说明这种架构,其中深度学习和数据工程部分协同工作。
英文摘要:
Data-intensive applications are becoming commonplace in all science disciplines. They are comprised of a rich set of sub-domains such as data engineering, deep learning, and machine learning. These applications are built around efficient data abstractions and operators that suit the applications of different domains. Often lack of a clear definition of data structures and operators in the field has led to other implementations that do not work well together. The HPTMT architecture that we proposed recently, identifies a set of data structures, operators, and an execution model for creating rich data applications that links all aspects of data engineering and data science together efficiently. This paper elaborates and illustrates this architecture using an end-to-end application with deep learning and data engineering parts working together.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。