DiNoDB：一个交互式的临时数据自组织查询引擎
DiNoDB: an Interactive-speed Query Engine for Ad-hoc Queries on Temporary Data
摘要：
随着数据集规模的增长，分析应用程序很难即时洞察大型数据集。现代应用程序涉及大量数据的繁重批处理工作，同时需要对临时数据进行高效的自组织交互式分析。然而，现有的解决方案通常集中在这两个方面之一，在很大程度上忽略了两者之间协同增效的需要。因此，交互式查询需要重新遍历整个数据集（例如，数据加载），其可以仅在长时间段内查询数据时提供有意义的投资回报。在本文中，我们提出了DiNoDB，一个交互式的速度查询引擎临时数据的临时查询。DiNoDB避免了传统RDBMS和当前交互式分析解决方案所特有的昂贵的加载和转换阶段。它专为机器学习和数据探索用例中的现代工作流量身定制，这些用例通常涉及对数据进行批处理和交互式分析的循环迭代，这通常适用于狭窄的处理窗口。DiNoDB的关键创新是在批处理阶段创建元数据，DiNoDB利用元数据加快交互式查询。我们的实验分析表明，DiNoDB实现了很好的性能，为广泛的ad-hoc查询相比，替代%，如Hive，Stado，SparkSQL和Impala。
英文摘要：
As data sets grow in size, analytics applications struggle to get instant insight into large datasets. Modern applications involve heavy batch processing jobs over large volumes of data and at the same time require efficient ad-hoc interactive analytics on temporary data. Existing solutions, however, typically focus on one of these two aspects, largely ignoring the need for synergy between the two. Consequently, interactive queries need to re-iterate costly passes through the entire dataset (e.g., data loading) that may provide meaningful return on investment only when data is queried over a long period of time. In this paper, we propose DiNoDB, an interactive-speed query engine for ad-hoc queries on temporary data. DiNoDB avoids the expensive loading and transformation phase that characterizes both traditional RDBMSs and current interactive analytics solutions. It is tailored to modern workflows found in machine learning and data exploration use cases, which often involve iterations of cycles of batch and interactive analytics on data that is typically useful for a narrow processing window. The key innovation of DiNoDB is to piggyback on the batch processing phase the creation of metadata that DiNoDB exploits to expedite the interactive queries. Our experimental analysis demonstrates that DiNoDB achieves very good performance for a wide range of ad-hoc queries compared to alternatives %such as Hive, Stado, SparkSQL and Impala.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
