AFrame：扩展DataFrames用于大规模现代数据分析（扩展版）
AFrame: Extending DataFrames for Large-Scale Modern Data Analysis (Extended Version)
摘要：
分析当今日益庞大的数据量，可能包括自定义机器学习模型的应用，需要利用分布式框架。这可能会给“普通”数据科学家带来严重的生产力问题。本文介绍了AFrame，这是一种由大数据管理系统提供支持的新的可扩展数据分析包，它扩展了数据科学家熟悉的DataFrame操作，以有效地对大规模托管数据进行操作。AFrame作为Apache AsterixDB之上的一层实现，通过一个并行的、无共享的大型数据管理系统透明地扩展DataFrame操作和机器学习模型调用的执行。AFrame递增地构造SQL++查询，并利用AsterixDB的半结构化数据管理设施、用户定义函数支持和实时数据摄取支持。为了评估所提出的方法，本文还介绍了一个可扩展的微基准用于评估DataFrame的性能，在单节点和分布式设置，通过收集代表性的分析操作。本文介绍了AFrame的体系结构，描述了AsterixDB的底层功能，有效地支持现代数据分析操作，并利用建议的基准评估和比较的性能和支持大规模数据分析提供的替代DataFrame库。
英文摘要：
Analyzing the increasingly large volumes of data that are available today, possibly including the application of custom machine learning models, requires the utilization of distributed frameworks. This can result in serious productivity issues for "normal" data scientists. This paper introduces AFrame, a new scalable data analysis package powered by a Big Data management system that extends the data scientists' familiar DataFrame operations to efficiently operate on managed data at scale. AFrame is implemented as a layer on top of Apache AsterixDB, transparently scaling out the execution of DataFrame operations and machine learning model invocation through a parallel, shared-nothing big data management system. AFrame incrementally constructs SQL++ queries and leverages AsterixDB's semistructured data management facilities, user-defined function support, and live data ingestion support. In order to evaluate the proposed approach, this paper also introduces an extensible micro-benchmark for use in evaluating DataFrame performance in both single-node and distributed settings via a collection of representative analytic operations. This paper presents the architecture of AFrame, describes the underlying capabilities of AsterixDB that efficiently support modern data analytic operations, and utilizes the proposed benchmark to evaluate and compare the performance and support for large-scale data analyses provided by alternative DataFrame libraries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
