视野之外的生成：基于整体注意力的多视图驾驶场景视频生成
Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention
AI总结：
针对自动驾驶多视图视频生成的跨维度一致性难题，提出带整体4D注意力的CogDriving网络、轻量Micro-Controller及重加权学习目标，在nuScenes数据集上取得37.8的FVD得分。
摘要：
为自动驾驶训练生成多视图视频近年来备受关注，其核心挑战在于同时保证跨视图与跨帧的一致性。现有方法通常对空间、时间和视图维度采用解耦的注意力机制，但这类方法往往难以维持各维度间的一致性，尤其是在处理于不同时间、不同视点出现的快速移动目标时表现不佳。本文提出CogDriving，一种用于合成高质量多视图驾驶视频的新型网络。CogDriving采用带有整体4D注意力模块的Diffusion Transformer架构，可同时建立空间、时间和视点维度间的关联。我们还为CogDriving设计了一款轻量控制器——Micro-Controller，其参数量仅为标准ControlNet的1.1%，可实现对鸟瞰图（Bird's-Eye-View）布局的精准控制。为提升自动驾驶关键目标实例的生成效果，我们提出一种重加权学习目标，可在训练过程中动态调整目标实例的学习权重。CogDriving在nuScenes验证集上表现优异，FVD得分达37.8，充分证明其生成真实驾驶视频的能力。项目主页为：https://luhannan.github.io/CogDrivingPage/。
英文摘要：
Generating multi-view videos for autonomous driving training has recently gained much attention, with the challenge of addressing both cross-view and cross-frame consistency. Existing methods typically apply decoupled attention mechanisms for spatial, temporal, and view dimensions. However, these approaches often struggle to maintain consistency across dimensions, particularly when handling fast-moving objects that appear at different times and viewpoints. In this paper, we present CogDriving, a novel network designed for synthesizing high-quality multi-view driving videos. CogDriving leverages a Diffusion Transformer architecture with holistic-4D attention modules, enabling simultaneous associations across the spatial, temporal, and viewpoint dimensions. We also propose a lightweight controller tailored for CogDriving, i.e., Micro-Controller, which uses only 1.1% of the parameters of the standard ControlNet, enabling precise control over Bird's-Eye-View layouts. To enhance the generation of object instances crucial for autonomous driving, we propose a re-weighted learning objective, dynamically adjusting the learning weights for object instances during training. CogDriving demonstrates strong performance on the nuScenes validation set, achieving an FVD score of 37.8, highlighting its ability to generate realistic driving videos. The project can be found at https://luhannan.github.io/CogDrivingPage/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
