arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.14902cs.DBcs.LG
收藏

Hydro:ML查询的自适应查询处理

Hydro: Adaptive Query Processing of ML Queries

Georgia Institute of Technology(佐治亚理工学院)

Gaurav Tarlok Kakkar, Jiashen Cao, Aubhro Sengupta, Joy Arulraj, Hyesoon Kim

AI总结:

本文提出Hydro,一个利用自适应查询处理(AQP)的ML中心DBMS,通过动态监控UDF统计信息、优化谓词评估顺序和资源分配,解决了ML查询中UDF性能瓶颈和数据依赖的计划优化问题,实现高达11.52倍加速。

摘要:

关系型数据库管理系统(DBMSs)中的查询优化对于快速查询处理至关重要。查询优化器依赖精确的选择率和成本估计,以在执行前有效地优化查询。虽然这种策略对关系型DBMSs有效,但对于专为处理机器学习(ML)查询而定制的DBMSs来说并不充分。在以ML为中心的DBMSs中,查询优化因两个原因而具有挑战性。首先,查询的性能瓶颈转移到通常包装深度学习模型的用户定义函数(UDFs)上,使得在不分析查询的情况下难以准确估计UDF统计信息。这导致不准确的统计信息和次优的查询计划。其次,ML查询的最优查询计划是数据依赖的,需要DBMSs在执行期间动态调整查询计划。因此,静态查询计划对于此类查询是不充分的。在本文中,我们提出了Hydro,一个利用自适应查询处理(AQP)来高效处理ML查询的以ML为中心的DBMS。Hydro旨在通过确保最优的谓词评估顺序和提高UDF执行的可扩展性,来快速评估基于UDF的查询谓词。通过集成AQP,Hydro持续监控UDF统计信息,以最优顺序将数据路由到谓词,并动态分配资源以评估谓词。我们通过四个说明性用例展示了Hydro的有效性,与基线系统相比实现了高达11.52倍的加速。

英文摘要:

Query optimization in relational database management systems (DBMSs) is critical for fast query processing. The query optimizer relies on precise selectivity and cost estimates to effectively optimize queries prior to execution. While this strategy is effective for relational DBMSs, it is not sufficient for DBMSs tailored for processing machine learning (ML) queries. In ML-centric DBMSs, query optimization is challenging for two reasons. First, the performance bottleneck of the queries shifts to user-defined functions (UDFs) that often wrap around deep learning models, making it difficult to accurately estimate UDF statistics without profiling the query. This leads to inaccurate statistics and sub-optimal query plans. Second, the optimal query plan for ML queries is data-dependent, necessitating DBMSs to adapt the query plan on the fly during execution. So, a static query plan is not sufficient for such queries. In this paper, we present Hydro, an ML-centric DBMS that utilizes adaptive query processing (AQP) for efficiently processing ML queries. Hydro is designed to quickly evaluate UDF-based query predicates by ensuring optimal predicate evaluation order and improving the scalability of UDF execution. By integrating AQP, Hydro continuously monitors UDF statistics, routes data to predicates in an optimal order, and dynamically allocates resources for evaluating predicates. We demonstrate Hydro's efficacy through four illustrative use cases, delivering up to 11.52x speedup over a baseline system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑