arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09447cs.DB
收藏

多维点的快速自适应批量加载

Fast and Adaptive Bulk Loading of Multidimensional Points

Moin Hussain Moti, Dimitris Papadias

AI总结:

针对多维点批量加载依赖多次外部排序的问题,提出基于线性扫描的FMBI及利用查询工作负载的自适应AMBI,并扩展至分布式并行处理,显著降低构建与查询综合成本。

摘要:

现有的基于磁盘的多维点批量加载方法涉及多次外部排序应用。本文提出采用线性扫描的技术,因此速度显著更快。所得FMBI索引具备若干理想特性,包括近乎全满且方正的节点、零重叠,并具有出色的查询性能。作为第二项贡献,我们开发了自适应版本AMBI,它利用查询工作负载,仅为包含查询结果的数据空间部分构建部分索引。最后,我们将FMBI和AMBI扩展到分布式系统中的并行批量加载与查询处理。使用真实数据集进行的大量实验评估证实,FMBI和AMBI在索引构建与查询处理的综合成本方面明显优于竞争方法,有时优势达到数个数量级。

英文摘要:

Existing methods for bulk loading disk-based multidimensional points involve multiple applications of external sorting. In this paper, we propose techniques that apply linear scan, and are therefore significantly faster. The resulting FMBI Index possesses several desirable properties, including almost full and square nodes with zero overlap, and has excellent query performance. As a second contribution, we develop an adaptive version AMBI, which utilizes the query workload to build a partial index only for parts of the data space that contain query results. Finally, we extend FMBI and AMBI to parallel bulk loading and query processing in distributed systems. An extensive experimental evaluation with real datasets confirms that FMBI and AMBI clearly outperform competitors in terms of combined index construction and query processing cost, sometimes by orders of magnitude.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑