高维数据的一致和灵活选择性估计
Consistent and Flexible Selectivity Estimation for High-Dimensional Data
摘要：
选择性估计的目的是估计满足选择标准的数据库对象的数量。准确、有效地解决这个问题对于密度估计、离群点检测、查询优化和数据集成等许多应用都是必不可少的。估计问题对于大规模高维数据尤其具有挑战性，这是由于维数灾难、不同查询之间的选择性的大方差以及使估计器一致的需要（即，选择性在阈值中是不减小的）。我们提出了一种新的基于深度学习的模型，该模型学习查询相关的分段线性函数作为选择性估计器，该模型可以灵活地拟合任何距离函数和查询对象的选择性曲线，同时保证输出在阈值内不减。为了提高大型数据集的准确性，我们建议将数据集划分为多个不相交的子集，并在每个子集上构建局部模型。我们在真实数据集上进行了实验，结果表明，该模型在准确性方面始终优于最先进的模型，并且对实际应用非常有用。
英文摘要：
Selectivity estimation aims at estimating the number of database objects that satisfy a selection criterion. Answering this problem accurately and efficiently is essential to many applications, such as density estimation, outlier detection, query optimization, and data integration. The estimation problem is especially challenging for large-scale high-dimensional data due to the curse of dimensionality, the large variance of selectivity across different queries, and the need to make the estimator consistent (i.e., the selectivity is non-decreasing in the threshold). We propose a new deep learning-based model that learns a query-dependent piecewise linear function as selectivity estimator, which is flexible to fit the selectivity curve of any distance function and query object, while guaranteeing that the output is non-decreasing in the threshold. To improve the accuracy for large datasets, we propose to partition the dataset into multiple disjoint subsets and build a local model on each of them. We perform experiments on real datasets and show that the proposed model consistently outperforms state-of-the-art models in accuracy in an efficient way and is useful for real applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
