arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18151cs.CV
收藏

SynRS3D:面向单目遥感影像全球3D语义理解的合成数据集

SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery

The University of Tokyo(东京大学);RIKEN AIP(理化学研究所人工智能项目)

Jian Song, Hongruixuan Chen, Weihao Xuan, Junshi Xia, Naoto Yokoya

AI总结:

针对单视图遥感影像全球3D语义理解面临的数据获取与标注成本高的挑战,构建了最大合成遥感3D数据集SynRS3D,并提出多任务无监督域适应方法RS3DAda,有效实现从合成到真实场景的迁移及全球单目3D理解。

摘要:

从单视图高分辨率遥感(RS)影像中进行全球语义3D理解对地球观测(EO)至关重要。然而,由于标注和数据收集的高昂成本,以及数据可用性的地理限制,该任务面临重大挑战。为应对这些挑战,合成数据因易于获取,从而能够提供大规模且多样化的数据集,成为一种极具前景的解决方案。我们开发了一种专为EO设计的合成数据生成管线,并推出了最大的合成遥感3D数据集SynRS3D。SynRS3D包含69,667张高分辨率光学图像,覆盖全球六种不同城市风格,具备八种土地覆盖类型、精确的高度信息以及建筑变化掩膜。为进一步提升其实用性,我们开发了一种新颖的多任务无监督域适应(UDA)方法RS3DAda,并与我们的合成数据集相结合,该方法促进了土地覆盖制图和高度估计任务中从合成场景到真实场景的遥感特定转换,最终实现基于合成数据的全球单目3D语义理解。在多个真实世界数据集上的大量实验证明了我们合成数据集及所提出的RS3DAda方法的适应性和有效性。SynRS3D及相关代码将被公开。

英文摘要:

Global semantic 3D understanding from single-view high-resolution remote sensing (RS) imagery is crucial for Earth Observation (EO). However, this task faces significant challenges due to the high costs of annotations and data collection, as well as geographically restricted data availability. To address these challenges, synthetic data offer a promising solution by being easily accessible and thus enabling the provision of large and diverse datasets. We develop a specialized synthetic data generation pipeline for EO and introduce SynRS3D, the largest synthetic RS 3D dataset. SynRS3D comprises 69,667 high-resolution optical images that cover six different city styles worldwide and feature eight land cover types, precise height information, and building change masks. To further enhance its utility, we develop a novel multi-task unsupervised domain adaptation (UDA) method, RS3DAda, coupled with our synthetic dataset, which facilitates the RS-specific transition from synthetic to real scenarios for land cover mapping and height estimation tasks, ultimately enabling global monocular 3D semantic understanding based on synthetic data. Extensive experiments on various real-world datasets demonstrate the adaptability and effectiveness of our synthetic dataset and proposed RS3DAda method. SynRS3D and related codes will be available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑