SafeMVDrive：真实世界域中的多视角安全关键驾驶视频合成
SafeMVDrive: Multi-view Safety-Critical Driving Video Synthesis in the Real World Domain
AI总结：
SafeMVDrive提出首个真实世界域多视角安全关键驾驶视频合成框架，结合轨迹生成与扩散视频生成，显著提升端到端规划器碰撞率测试效果。
摘要：
安全关键场景虽然罕见，但对于评估和增强自动驾驶系统的鲁棒性至关重要。现有方法能够生成安全关键的驾驶轨迹、仿真或单视角视频，但无法满足先进端到端自动驾驶系统（E2E AD）的需求，这些系统需要真实世界的多视角视频数据。为弥补这一差距，我们提出了SafeMVDrive，这是首个旨在生成基于真实世界域的高质量、安全关键、多视角驾驶视频的框架。SafeMVDrive策略性地将安全关键轨迹生成器与先进的多视角视频生成器相结合。为应对这一整合中的挑战，我们首先通过引入视觉上下文（此类生成器此前无法获取）来增强轨迹生成器的场景理解能力，并利用GRPO微调的视觉语言模型实现更真实且上下文感知的轨迹生成。其次，鉴于现有多视角视频生成器难以渲染逼真的碰撞事件，我们引入了一种两阶段可控轨迹生成机制，生成碰撞规避轨迹，确保视频质量和安全关键保真度。最后，我们采用基于扩散的多视角视频生成器，从生成的轨迹合成高质量的安全关键驾驶视频。在端到端自动驾驶规划器上进行的实验表明，使用我们生成的数据进行测试时，碰撞率显著增加，验证了SafeMVDrive在压力测试规划模块方面的有效性。我们的代码、示例和数据集可在 https://zhoujiawei3.github.io/SafeMVDrive/ 公开获取。
英文摘要：
Safety-critical scenarios are rare yet pivotal for evaluating and enhancing the robustness of autonomous driving systems. While existing methods generate safety-critical driving trajectories, simulations, or single-view videos, they fall short of meeting the demands of advanced end-to-end autonomous systems (E2E AD), which require real-world, multi-view video data. To bridge this gap, we introduce SafeMVDrive, the first framework designed to generate high-quality, safety-critical, multi-view driving videos grounded in real-world domains. SafeMVDrive strategically integrates a safety-critical trajectory generator with an advanced multi-view video generator. To tackle the challenges inherent in this integration, we first enhance scene understanding ability of the trajectory generator by incorporating visual context -- which is previously unavailable to such generator -- and leveraging a GRPO-finetuned vision-language model to achieve more realistic and context-aware trajectory generation. Second, recognizing that existing multi-view video generators struggle to render realistic collision events, we introduce a two-stage, controllable trajectory generation mechanism that produces collision-evasion trajectories, ensuring both video quality and safety-critical fidelity. Finally, we employ a diffusion-based multi-view video generator to synthesize high-quality safety-critical driving videos from the generated trajectories. Experiments conducted on an E2E AD planner demonstrate a significant increase in collision rate when tested with our generated data, validating the effectiveness of SafeMVDrive in stress-testing planning modules. Our code, examples, and datasets are publicly available at: https://zhoujiawei3.github.io/SafeMVDrive/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
