查询驱动的机器学习模型已作为一种有前途的查询选择性估计技术出现。然而，从理论角度看，这些技术的有效性仍知之甚少，因为实际解决方案与基于Probably Approximately Correct (PAC) 学习框架的最先进理论之间存在显著差距。本文旨在弥合理论与实践之间的差距。首先，我们证明由符号测度诱导的选择性预测器是可学习的，这放松了PAC理论对概率测度的依赖。更重要的是，在此基础上，我们建立了在温和假设下，此类选择性预测器在分布外（OOD）泛化误差界上的有利表现。这些理论进步为我们提供了对查询驱动选择性学习的分布内和分布外泛化能力的更好理解，并促进了两种改进分布外泛化的通用策略的设计。我们实证验证了我们的技术在预测准确性和查询延迟性能方面显著帮助查询驱动选择性模型泛化到分布外查询，同时保持其优越的分布内泛化性能。
英文摘要：
Query-driven machine learning models have emerged as a promising estimation technique for query selectivities. Yet, surprisingly little is known about the efficacy of these techniques from a theoretical perspective, as there exist substantial gaps between practical solutions and state-of-the-art (SOTA) theory based on the Probably Approximately Correct (PAC) learning framework. In this paper, we aim to bridge the gaps between theory and practice. First, we demonstrate that selectivity predictors induced by signed measures are learnable, which relaxes the reliance on probability measures in SOTA theory. More importantly, beyond the PAC learning framework (which only allows us to characterize how the model behaves when both training and test workloads are drawn from the same distribution), we establish, under mild assumptions, that selectivity predictors from this class exhibit favorable out-of-distribution (OOD) generalization error bounds.
These theoretical advances provide us with a better understanding of both the in-distribution and OOD generalization capabilities of query-driven selectivity learning, and facilitate the design of two general strategies to improve OOD generalization for existing query-driven selectivity models. We empirically verify that our techniques help query-driven selectivity models generalize significantly better to OOD queries both in terms of prediction accuracy and query latency performance, while maintaining their superior in-distribution generalization performance.
A Generalization Theory based on Independent and Task-Identically Distributed Assumption
University of Science and Technology of China（中国科学技术大学）；Beijing Jiaotong University（北京交通大学）；Chinese Academy of Sciences（中国科学院）；Institute of Automation, CAS（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
A General Framework for the Practical Disintegration of PAC-Bayesian Bounds
Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL Research University（巴黎文理研究大学）；Université Laval（拉瓦尔大学）；Université Jean Monnet Saint-Étienne（圣艾蒂安让·莫内大学）；Institut d Optique Graduate School（光学研究所研究生院）；Institut Universitaire de France（法兰西大学研究院）
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Tokyo Institute of Technology（东京工业大学）；AIST-Tokyo Tech RWBC-OIL, AIST（产业技术综合研究所-东京工业大学计算生物学与复杂系统研究中心，产业技术综合研究所）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；NVIDIA（英伟达）
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Samsung Medical Center（三星医疗中心）；Sungkyunkwan University School of Medicine（成均馆大学医学院）；AITRICS
Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；INRIA（法国国家信息与自动化研究所）；Imperial College London（伦敦帝国学院）；CNRS, Ecole Normale Supérieure, PSL Research University（法国国家科学研究中心、巴黎高等师范学院、巴黎文理研究大学）
