利用 BEV 范式进行地对空图像合成
Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesis
AI总结：
提出 SkyDiffusion，通过 Curved-BEV 转换与 BEV 引导扩散模型从街景生成内容一致的航拍图像，并构建 Ground2Aerial-3 数据集，在多类跨视图基准上超越现有方法。
摘要：
地对空图像合成专注于根据相应的地面街景图像生成逼真的航拍图像，同时保持一致的内容布局，模拟俯视视角。显著的视角差异导致不同视图之间存在域差距，而密集城市场景限制了街景的可见范围，使这一跨视图生成任务尤其具有挑战性。本文提出 SkyDiffusion，这是一种利用扩散模型和 Bird's-Eye View（BEV，鸟瞰图）范式、从街景图像合成航拍图像的新型跨视图生成方法。SkyDiffusion 中的 Curved-BEV 方法将街景图像转换为 BEV 视角，有效弥合域差距，并采用“多对一”映射策略解决密集城市场景中的遮挡问题。随后，SkyDiffusion 设计了 BEV 引导的扩散模型，以生成内容一致且逼真的航拍图像。此外，我们引入了一个名为 Ground2Aerial-3 的新数据集，面向多种地对空图像合成应用，包括灾场航拍合成、低空 UAV 图像合成以及历史高分辨率卫星图像合成任务。实验结果表明，SkyDiffusion 在自然（CVUSA）、郊区（CVACT）、城市（VIGOR-Chicago）以及多种应用场景（G2A-3）的跨视图数据集上优于最先进方法，实现了逼真且内容一致的航拍图像生成。本工作的代码、数据集和更多信息可在 https://opendatalab.github.io/skydiffusion/ 获取。
英文摘要：
Ground-to-aerial image synthesis focuses on generating realistic aerial images from corresponding ground street view images while maintaining consistent content layout, simulating a top-down view. The significant viewpoint difference leads to domain gaps between views, and dense urban scenes limit the visible range of street views, making this cross-view generation task particularly challenging. In this paper, we introduce SkyDiffusion, a novel cross-view generation method for synthesizing aerial images from street view images, utilizing a diffusion model and the Bird's-Eye View (BEV) paradigm. The Curved-BEV method in SkyDiffusion converts street-view images into a BEV perspective, effectively bridging the domain gap, and employs a "multi-to-one" mapping strategy to address occlusion issues in dense urban scenes. Next, SkyDiffusion designed a BEV-guided diffusion model to generate content-consistent and realistic aerial images. Additionally, we introduce a novel dataset, Ground2Aerial-3, designed for diverse ground-to-aerial image synthesis applications, including disaster scene aerial synthesis, low-altitude UAV image synthesis, and historical high-resolution satellite image synthesis tasks. Experimental results demonstrate that SkyDiffusion outperforms state-of-the-art methods on cross-view datasets across natural (CVUSA), suburban (CVACT), urban (VIGOR-Chicago), and various application scenarios (G2A-3), achieving realistic and content-consistent aerial image generation. The code, datasets and more information of this work can be found at https://opendatalab.github.io/skydiffusion/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
