arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.13795cs.DB
收藏

用于AIS数据的分布式空间数据仓库(DIPAAL)

A Distributed Spatial Data Warehouse for AIS Data (DIPAAL)

Aalborg University(奥尔堡大学)

Alex S. Klitgaard, Lau E. Josefsen, Mikael V. Mikkelsen, Kristian Torp

AI总结:

本文提出了一种用于AIS数据的分布式空间数据仓库,通过空间划分和栅格查询方法提升大规模船舶轨迹分析效率。

摘要:

船舶的AIS数据非常适合分析单艘船的运动并监控特定区域内的所有船只。然而,在可使用之前,AIS数据需要进行清理、处理和存储。本文提出了一种高效的、模块化的ETL流程用于加载AIS数据,以及一个分布式空间数据仓库,用于存储船只的轨迹。为了高效分析大量船只,本文提出了一种基于栅格的方法来查询AIS数据。设计、开发并评估了一个空间划分的数据仓库,具有细粒度的单元表示和热力图展示。目前,数据仓库存储了约312亿公里的船舶轨迹,以及在最大表中超过80亿行数据。发现搜索单元表示比搜索轨迹表示更快。进一步,我们展示了空间划分的分片能够为大范围内的单元和热力图分析提供一致的良好扩展性,扩展范围在354%至1164%之间,当工作线程增加5倍时。

英文摘要:

AIS data from ships is excellent for analyzing single-ship movements and monitoring all ships within a specific area. However, the AIS data needs to be cleaned, processed, and stored before being usable. This paper presents a system consisting of an efficient and modular ETL process for loading AIS data, as well as a distributed spatial data warehouse storing the trajectories of ships. To efficiently analyze a large set of ships, a raster approach to querying the AIS data is proposed. A spatially partitioned data warehouse with a granularized cell representation and heatmap presentation is designed, developed, and evaluated. Currently the data warehouse stores ~312 million kilometers of ship trajectories and more than +8 billion rows in the largest table. It is found that searching the cell representation is faster than searching the trajectory representation. Further, we show that the spatially divided shards enable a consistently good scale-up for both cell and heatmap analytics in large areas, ranging between 354% to 1164% with a 5x increase in workers

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑