一种有效索引学习的惰性方法
A Lazy Approach for Efficient Index Learning
摘要:
使用神经网络学习索引已被证明优于传统的索引,如B树在查询时间和内存。然而,学习大型数据集的分布可能是昂贵的,并且更新学习的索引是困难的,从而阻碍了它们在实际应用中的使用。在本文中,我们通过敏捷模型重用来解决学习索引的效率和更新问题。我们在一组合成(而不是真实)数据集上预训练学习的索引,并提出了一种新的方法来将这些预训练的模型重新用于新的(真实)数据集。创建合成数据集以覆盖大范围的不同分布。给定一个新的数据集DT,我们选择一个类似于DT的合成数据集的学习索引来索引DT。当选择预训练索引时,我们显示了索引错误的界限。我们进一步展示了我们的技术如何处理数据更新和绑定由此产生的索引错误。合成和真实数据集上的实验结果证实了我们提出的懒惰(模型重用)方法的有效性和效率。
英文摘要:
Learned indices using neural networks have been shown to outperform traditional indices such as B-trees in both query time and memory. However, learning the distribution of a large dataset can be expensive, and updating learned indices is difficult, thus hindering their usage in practical applications. In this paper, we address the efficiency and update issues of learned indices through agile model reuse. We pre-train learned indices over a set of synthetic (rather than real) datasets and propose a novel approach to reuse these pre-trained models for a new (real) dataset. The synthetic datasets are created to cover a large range of different distributions. Given a new dataset DT, we select the learned index of a synthetic dataset similar to DT, to index DT. We show a bound over the indexing error when a pre-trained index is selected. We further show how our techniques can handle data updates and bound the resultant indexing errors. Experimental results on synthetic and real datasets confirm the effectiveness and efficiency of our proposed lazy (model reuse) approach.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。