GEqO：机器学习加速的语义等价性检测
GEqO: ML-Accelerated Semantic Equivalence Detection
AI总结：
本文提出基于机器学习的轻量级框架 GEqO，通过双过滤器快速剔除不等价子表达式、半监督反馈循环改进模型及数据库无关特征化方法，实现大规模语义等价性检测，性能较现有方法最高提升200倍。
摘要：
大规模分析引擎已成为现代数据驱动型企业获取商业洞察和驱动行动的核心依赖。这些引擎支持每天处理海量数据的大量分析作业，且工作负载中常充斥着多个作业间的重叠计算。重用公共计算对于高效的集群资源利用和减少作业执行时间至关重要。检测公共计算是减少这种计算冗余的首要且关键的步骤。然而，在大规模分析引擎上检测等价性需要完全自动化且高效、可扩展的解决方案。此外，为了最大化计算重用，需要在语义层面而非仅仅在语法层面检测等价性（即检测看似不同的查询的语义等价性的能力）。遗憾的是，现有解决方案无法满足这些要求。在本文中，我们通过提出 GEqO 填补了这一空白，这是一个便携且轻量级的基于机器学习的框架，可大规模高效识别语义等价的计算。GEqO 引入了两个基于机器学习的过滤器，可快速剔除不等价的子表达式，并采用半监督学习反馈循环，通过智能采样机制迭代改进其模型。此外，凭借其新颖的数据库无关特征化方法，GEqO 能够将学习成果从一个工作负载和数据库迁移到另一个。我们广泛的实证评估表明，在类 TPC-DS 查询上，GEqO 带来了显著的性能提升——比自动验证器快达 200 倍——并且比基于优化器和签名的等价性检测方法发现了多达 2 倍的等价性。
英文摘要：
Large scale analytics engines have become a core dependency for modern data-driven enterprises to derive business insights and drive actions. These engines support a large number of analytic jobs processing huge volumes of data on a daily basis, and workloads are often inundated with overlapping computations across multiple jobs. Reusing common computation is crucial for efficient cluster resource utilization and reducing job execution time. Detecting common computation is the first and key step for reducing this computational redundancy. However, detecting equivalence on large-scale analytics engines requires efficient and scalable solutions that are fully automated. In addition, to maximize computation reuse, equivalence needs to be detected at the semantic level instead of just the syntactic level (i.e., the ability to detect semantic equivalence of seemingly different-looking queries). Unfortunately, existing solutions fall short of satisfying these requirements. In this paper, we take a major step towards filling this gap by proposing GEqO, a portable and lightweight machine-learning-based framework for efficiently identifying semantically equivalent computations at scale. GEqO introduces two machine-learning-based filters that quickly prune out nonequivalent subexpressions and employs a semi-supervised learning feedback loop to iteratively improve its model with an intelligent sampling mechanism. Further, with its novel database-agnostic featurization method, GEqO can transfer the learning from one workload and database to another. Our extensive empirical evaluation shows that, on TPC-DS-like queries, GEqO yields significant performance gains-up to 200x faster than automated verifiers-and finds up to 2x more equivalences than optimizer and signature-based equivalence detection approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
