DriveEditor：面向驾驶场景可控物体编辑的统一三维信息引导框架
DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes
AI总结：
DriveEditor提出统一的扩散框架，通过三维边界框位置控制和单参考图像外观保持模块，实现驾驶视频中物体的重新定位、替换、删除和插入，并在nuScenes上验证了高保真可控编辑能力。
摘要：
以视觉为中心的自动驾驶系统需要多样化的数据来进行稳健的训练和评估，这可以通过在现有场景捕获中操纵物体位置和外观来增强。尽管扩散模型的最新进展在视频编辑中展现出潜力，但将其应用于驾驶场景中的物体操纵仍然具有挑战性，原因在于位置控制不够精确，以及难以保持高保真的物体外观。为了解决这些在位置和外观控制方面的挑战，我们提出了DriveEditor，一个基于扩散模型的驾驶视频物体编辑框架。DriveEditor为全面的物体编辑操作提供了一个统一框架，包括重新定位、替换、删除和插入。这些多样化的操作均通过一组共享的不同输入来实现，并由相同的位置控制模块和外观保持模块处理。位置控制模块在保留深度信息的同时投影给定的三维边界框，并将其分层注入扩散过程，从而实现对物体位置和朝向的精确控制。外观保持模块通过采用三层方法，利用单张参考图像保持一致的属性：低层细节保留、高层语义维护，以及来自新视角合成模型的三维先验集成。在nuScenes数据集上进行的大量定性和定量评估表明，DriveEditor在生成多样化的驾驶场景编辑方面具有卓越的保真度和可控性，并且具备显著促进下游任务的能力。项目主页：https://yvanliang.github.io/DriveEditor。
英文摘要：
Vision-centric autonomous driving systems require diverse data for robust training and evaluation, which can be augmented by manipulating object positions and appearances within existing scene captures. While recent advancements in diffusion models have shown promise in video editing, their application to object manipulation in driving scenarios remains challenging due to imprecise positional control and difficulties in preserving high-fidelity object appearances. To address these challenges in position and appearance control, we introduce DriveEditor, a diffusion-based framework for object editing in driving videos. DriveEditor offers a unified framework for comprehensive object editing operations, including repositioning, replacement, deletion, and insertion. These diverse manipulations are all achieved through a shared set of varying inputs, processed by identical position control and appearance maintenance modules. The position control module projects the given 3D bounding box while preserving depth information and hierarchically injects it into the diffusion process, enabling precise control over object position and orientation. The appearance maintenance module preserves consistent attributes with a single reference image by employing a three-tiered approach: low-level detail preservation, high-level semantic maintenance, and the integration of 3D priors from a novel view synthesis model. Extensive qualitative and quantitative evaluations on the nuScenes dataset demonstrate DriveEditor's exceptional fidelity and controllability in generating diverse driving scene edits, as well as its remarkable ability to facilitate downstream tasks. Project page: https://yvanliang.github.io/DriveEditor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
