COAX:基于软函数相关性的多维数据相关性索引
COAX: Correlation-Aware Indexing on Multidimensional Data with Soft Functional Dependencies
摘要:
最近的工作提出了学习索引结构,它学习底层数据集的分布以提高性能。学习索引的初步工作表明,通过学习数据的累积分布函数,B树等索引结构可以将其性能提高一个数量级,同时具有更小的内存占用。在本文中,我们提出了COAX,这是一种多维数据的学习索引,它不是学习键的分布,而是学习数据集属性之间的相关性。我们的方法是由观察驱动的,在许多数据集中,两个(或多个)属性的值是相关的。COAX利用这些相关性来降低数据集的维度。更准确地说,我们学习了如何从剩余的属性中推断出一个(或多个)属性$C_d$,因此不再需要索引属性$C_d$。这降低了维度,因此使索引更小,更有效。我们从理论上研究所提出的技术的有效性的基础上的FD属性的可预测性。我们进一步通过实验表明,通过预测数据中的相关属性,我们可以提高查询执行时间,减少索引的内存开销。在我们的实验中,我们将执行时间减少了25%,同时将索引的内存占用减少了四个数量级。
英文摘要:
Recent work proposed learned index structures, which learn the distribution of the underlying dataset to improve performance. The initial work on learned indexes has shown that by learning the cumulative distribution function of the data, index structures such as the B-Tree can improve their performance by one order of magnitude while having a smaller memory footprint. In this paper, we present COAX, a learned index for multidimensional data that, instead of learning the distribution of keys, learns the correlations between attributes of the dataset. Our approach is driven by the observation that in many datasets, values of two (or multiple) attributes are correlated. COAX exploits these correlations to reduce the dimensionality of the datasets. More precisely, we learn how to infer one (or multiple) attribute $C_d$ from the remaining attributes and hence no longer need to index attribute $C_d$. This reduces the dimensionality and hence makes the index smaller and more efficient. We theoretically investigate the effectiveness of the proposed technique based on the predictability of the FD attributes. We further show experimentally that by predicting correlated attributes in the data, we can improve the query execution time and reduce the memory overhead of the index. In our experiments, we reduce the execution time by 25% while reducing the memory footprint of the index by four orders of magnitude.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。