云数据仓库中预执行查询槽时间预测：一种基于特征范围的机器学习方法
Pre-Execution Query Slot-Time Prediction in Cloud Data Warehouses: A Feature-Scoped Machine Learning Approach
AI总结：
本文提出一种基于特征范围的机器学习方法，通过预执行时可观察的信号预测BigQuery槽时间，减少预算超支和调度退化，实验显示在成本显著查询上模型性能优于基线方法。
摘要：
云数据仓库根据消耗的槽时间计费。在共享多租户环境中，查询成本高度变化且难以在执行前估计，导致预算超支和调度降级。静态查询规划器启发式方法无法捕捉复杂的SQL结构、数据偏斜和工作负载竞争。本文提出一种基于特征范围的机器学习方法，利用预执行时可观察的信号预测BigQuery槽时间：结构化的查询复杂度分数，来源于SQL操作符成本、规划器估计的数据体积特征以及工作负载元数据，以及查询文本的文本特征。模型使用HistGradientBoostingRegressor训练在对数转换的槽时间上，结合TF-IDF + TruncatedSVD-512文本流程与数值和类别特征。在七个部署环境中训练749个查询，并在两个保留环境中进行离分布评估，模型在完整工作负载上达到MAE 1.17槽分钟，RMSE 4.71，解释方差74%。在成本显著的查询（槽时间≥0.01分钟，N=282）上，模型达到MAE 3.10，优于预测均值基线（4.95）和预测中位数基线（4.54），减少30-37%。在长尾查询（≥20分钟，N=22）上，模型未优于简单基线，与假设一致，即长尾查询受当前特征范围外的不可观测运行时因素主导。描述了复杂度导向的双模型架构作为实用改进，并识别了缩小长尾差距的方向作为未来工作。
英文摘要：
Cloud data warehouses bill compute based on slot-time consumed. In shared multi-tenant environments, query cost is highly variable and hard to estimate before execution, causing budget overruns and degraded scheduling. Static query-planner heuristics fail to capture complex SQL structure, data skew, and workload contention. We present a feature-scoped machine learning approach that predicts BigQuery slot-time before execution using only pre-execution observable signals: a structured query complexity score derived from SQL operator costs, data volume features from planner estimates and workload metadata, and textual features from query text. We deliberately exclude runtime factors (slot-pool utilization, cache state, realized skew) unknowable at submission. The model uses a HistGradientBoostingRegressor trained on log-transformed slot-time, with a TF-IDF + TruncatedSVD-512 text pipeline fused with numeric and categorical features. Trained on 749 queries across seven deployment environments and evaluated out-of-distribution on 746 queries from two held-out environments, the model achieves MAE 1.17 slot-minutes, RMSE 4.71, and 74% explained variance on the full workload. On cost-significant queries (slot-time >= 0.01 min, N=282) the model achieves MAE 3.10 versus 4.95 for a predict-mean baseline and 4.54 for predict-median, a 30-37% reduction. On long-tail queries (>= 20 min, N=22) the model does not outperform trivial baselines, consistent with the hypothesis that long-tail queries are dominated by unobserved runtime factors outside the current feature scope. A complexity-routed dual-model architecture is described as a practical refinement, and directions for closing the long-tail gap are identified as future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
