arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06159cs.CV
收藏

驾驶用DINO:作为自动驾驶仿真到现实生成的统一桥梁的视觉基础特征

Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving

Technical University of Munich(慕尼黑技术大学);Huawei Hilbert Research Center(华为希利伯特研究中心);Huawei Riemann Lab(华为里曼实验室);Wuhan University(武汉大学);University of Science and Technology of China(中国科学技术大学);Nanjing University(南京大学)

Xuyang Chen, Conglang Zhang, Chuanheng Fu, Zihao Yang, Kaixuan Zhou, Yizhi Zhang, Yanfeng Zhang, Mingwei Sun, Zhen Dong, Xiaoxiao Long, Zengmao Wang, Liqiu Meng

AI总结:

DwD通过利用视觉基础模块特征作为统一桥梁,解决自动驾驶仿真到现实生成中的现实性与真实性困境,提升控制精度与时间稳定性。

摘要:

受可控视频扩散的出现驱动,现有的自动驾驶视频生成的Sim2Real方法通常依赖于显式的中间表示来弥合领域差距。然而,这些模态面临一个根本的现实性与真实性困境。低级信号(例如边缘、模糊图像)确保精确控制,但通过

英文摘要:

Driven by the emergence of Controllable Video Diffusion, existing Sim2Real methods for autonomous driving video generation typically rely on explicit intermediate representations to bridge the domain gap. However, these modalities face a fundamental Consistency-Realism Dilemma. Low-level signals (e.g., edges, blurred images) ensure precise control but compromise realism by "baking in" synthetic artifacts, whereas high-level priors (e.g., depth, semantics, HDMaps) facilitate photorealism but lack the structural detail required for consistent guidance. In this work, we present Driving with DINO (DwD), a novel framework that leverages Vision Foundation Module (VFM) features as a unified bridge between the simulation and real-world domains. We first identify that these features encode a spectrum of information, from high-level semantics to fine-grained structure. To effectively utilize this, we employ Principal Subspace Projection to discard the high-frequency elements responsible for "texture baking," while concurrently introducing Random Channel Tail Drop to mitigate the structural loss inherent in rigid dimensionality reduction, thereby reconciling realism with control consistency. Furthermore, to fully leverage DINOv3's high-resolution capabilities for enhancing control precision, we introduce a learnable Spatial Alignment Module that adapts these high-resolution features to the diffusion backbone. Finally, we propose a Causal Temporal Aggregator employing causal convolutions to explicitly preserve historical motion context when integrating frame-wise DINO features, which effectively mitigates motion blur and guarantees temporal stability. Project page: https://albertchen98.github.io/DwD-project/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑