arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1209.6490cs.DB
收藏

大型多维数据库的空间索引

Spatial Indexing of Large Multidimensional Databases

István Csabai, Márton Trencséni, Géza Herczegh, László Dobos, Péter Józsa, Norbert Purger, Tamás Budavári, Alexander Szalay

摘要:

诸如大型天文测量之类的科学工作产生了TB级的数据库。这些通常是多维数据库,必须进行可视化和挖掘,以便找到有趣的对象或提取有意义的和定性的新关系。这些任务所需的许多统计算法在对内存中的小数据集进行操作时运行得相当快,但在对不适合内存的大型数据库进行操作时会出现明显的性能下降。我们利用新的软件技术来开发和评估快速多维索引方案,这些方案本质上遵循数据的底层,高度不均匀的分布:它们是分层的均匀网格索引,分层的二进制空间分区,以及数据的采样平面Voronoi细分。我们的工作数据库是斯隆数字巡天的5维星等空间,拥有超过2.7亿个数据点,我们表明这些技术可以大大加快数据采掘操作,例如通过实例查找类似对象,对对象进行分类或将大量模拟集与观测结果进行比较。我们还在开发与多维数据库交互的工具,并以自适应的方式以多种分辨率显示数据。

英文摘要:

Scientific endeavors such as large astronomical surveys generate databases on the terabyte scale. These, usually multidimensional databases must be visualized and mined in order to find interesting objects or to extract meaningful and qualitatively new relationships. Many statistical algorithms required for these tasks run reasonably fast when operating on small sets of in-memory data, but take noticeable performance hits when operating on large databases that do not fit into memory. We utilize new software technologies to develop and evaluate fast multidimensional indexing schemes that inherently follow the underlying, highly non-uniform distribution of the data: they are layered uniform grid indices, hierarchical binary space partitioning, and sampled flat Voronoi tessellation of the data. Our working database is the 5-dimensional magnitude space of the Sloan Digital Sky Survey with more than 270 million data points, where we show that these techniques can dramatically speed up data mining operations such as finding similar objects by example, classifying objects or comparing extensive simulation sets with observations. We are also developing tools to interact with the multidimensional database and visualize the data at multiple resolutions in an adaptive manner.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑