从鸟瞰图到街景：利用潜在扩散模型生成多样且条件对齐的图像
From Bird's-Eye to Street View: Crafting Diverse and Condition-Aligned Images with Latent Diffusion Model
AI总结：
本文提出一种从BEV地图生成多视角街景图像的框架，通过神经视角变换生成对齐的语义分割图，再以此作为条件微调潜在扩散模型，实现多样且条件一致的街景生成。
摘要：
我们探索鸟瞰图（BEV）生成任务，即将BEV地图转换为对应的多视角街景图像。BEV因其统一的空域表示有助于多传感器融合，对多种自动驾驶应用至关重要。从BEV地图生成准确的街景图像，对于刻画复杂交通场景和增强驾驶算法十分关键。与此同时，基于扩散的条件图像生成模型已展现出显著成果，能够生成多样、高质量且与条件对齐的结果。然而，这类模型的训练需要大量数据和计算资源。因此，探索如何针对特定条件生成任务微调Stable Diffusion等先进模型，成为一条有前景的研究路径。本文提出一个实用的BEV布局图像生成框架。我们的方法包含两个主要组件：神经视角变换和街景图像生成。神经视角变换阶段通过学习BEV与透视视图之间的形状对应关系，将BEV地图转换为对齐的多视角语义分割图。随后，街景图像生成阶段利用这些分割图作为条件，引导经过微调的潜在扩散模型。该微调过程确保视角一致性和风格一致性。我们的模型在交通场景中充分利用大型预训练扩散模型的生成能力，有效生成多样且条件一致的街景图像。
英文摘要：
We explore Bird's-Eye View (BEV) generation, converting a BEV map into its corresponding multi-view street images. Valued for its unified spatial representation aiding multi-sensor fusion, BEV is pivotal for various autonomous driving applications. Creating accurate street-view images from BEV maps is essential for portraying complex traffic scenarios and enhancing driving algorithms. Concurrently, diffusion-based conditional image generation models have demonstrated remarkable outcomes, adept at producing diverse, high-quality, and condition-aligned results. Nonetheless, the training of these models demands substantial data and computational resources. Hence, exploring methods to fine-tune these advanced models, like Stable Diffusion, for specific conditional generation tasks emerges as a promising avenue. In this paper, we introduce a practical framework for generating images from a BEV layout. Our approach comprises two main components: the Neural View Transformation and the Street Image Generation. The Neural View Transformation phase converts the BEV map into aligned multi-view semantic segmentation maps by learning the shape correspondence between the BEV and perspective views. Subsequently, the Street Image Generation phase utilizes these segmentations as a condition to guide a fine-tuned latent diffusion model. This finetuning process ensures both view and style consistency. Our model leverages the generative capacity of large pretrained diffusion models within traffic contexts, effectively yielding diverse and condition-coherent street view images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
