Mirage: 驾驶场景中光实且一致的资产编辑一步视频扩散
Mirage: One-Step Video Diffusion for Photorealistic and Coherent Asset Editing in Driving Scenes
AI总结：
Mirage提出了一种用于驾驶场景中光实且一致的资产编辑的一步视频扩散模型，通过引入时序无关的潜在特征和两阶段数据对齐策略，提升了视觉保真度和时间一致性。
摘要：
以视觉为中心的自动驾驶系统依赖于多样且可扩展的训练数据来实现稳健的性能。虽然视频对象编辑为数据增强提供了一条有希望的途径，但现有方法往往难以在保持高视觉保真度和时间一致性方面取得平衡。在本工作中，我们提出了Mirage，一种用于驾驶场景中光实且一致的资产编辑的一步视频扩散模型。Mirage建立在文本到视频扩散先验之上，以确保跨帧的时间一致性。然而，3D因果变分自编码器往往由于压缩而出现空间保真度下降，且直接将3D编码器特征传递到解码器层会破坏时间因果性。为了解决这个问题，我们向3D解码器注入了从预训练的2D编码器中提取的时序无关的潜在特征，以恢复细节同时保持因果结构。此外，由于场景物体和插入资产是在不同的目标下优化的，它们的高斯分布会出现分布不匹配，导致姿态错位。为缓解这一问题，我们引入了一种两阶段的数据对齐策略，结合粗略的3D对齐和精细的2D优化，从而提高对齐性并提供更清晰的监督。广泛的实验表明，Mirage在多种编辑场景中实现了高真实性和时间一致性。除了资产编辑，Mirage还可以泛化到其他视频到视频翻译任务，成为未来研究的可靠基准。我们的代码可在https://github.com/wm-research/mirage上获得。
英文摘要：
Vision-centric autonomous driving systems rely on diverse and scalable training data to achieve robust performance. While video object editing offers a promising path for data augmentation, existing methods often struggle to maintain both high visual fidelity and temporal coherence. In this work, we propose \textbf{Mirage}, a one-step video diffusion model for photorealistic and coherent asset editing in driving scenes. Mirage builds upon a text-to-video diffusion prior to ensure temporal consistency across frames. However, 3D causal variational autoencoders often suffer from degraded spatial fidelity due to compression, and directly passing 3D encoder features to decoder layers breaks temporal causality. To address this, we inject temporally agnostic latents from a pretrained 2D encoder into the 3D decoder to restore detail while preserving causal structures. Furthermore, because scene objects and inserted assets are optimized under different objectives, their Gaussians exhibit a distribution mismatch that leads to pose misalignment. To mitigate this, we introduce a two-stage data alignment strategy combining coarse 3D alignment and fine 2D refinement, thereby improving alignment and providing cleaner supervision. Extensive experiments demonstrate that Mirage achieves high realism and temporal consistency across diverse editing scenarios. Beyond asset editing, Mirage can also generalize to other video-to-video translation tasks, serving as a reliable baseline for future research. Our code is available at https://github.com/wm-research/mirage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
