arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.04887cs.CV

HorizonWeaver: 通用多级语义编辑用于驾驶场景

HorizonWeaver: Generalizable Multi-Level Semantic Editing for Driving Scenes

Simon Fraser University(西蒙菲莎大学);NEC Labs America(NEC美国实验室)

Mauricio Soroco, Francesco Pittaluga, Zaid Tasneem, Abhishek Aich, Bingbing Zhuang, Wuyang Chen, Manmohan Chandraker, Ziyu Jiang

AI总结:

本文提出HorizonWeaver,通过多级粒度、丰富高层语义和普遍领域转移解决驾驶场景编辑难题,构建大规模数据集并改进模型与训练方法,实现复杂驾驶场景的高质量指令驱动编辑。

摘要:

确保自动驾驶安全需要生成超越现实测试的逼真可控驾驶场景。现有指令引导图像编辑器在密集安全关键驾驶布局中表现不佳。本文提出HorizonWeaver,解决三个核心挑战:多级粒度要求在密集环境中保持对象和场景级一致性;丰富高层语义需在多样化对象中遵循详细指令;普遍领域转移需处理气候、布局和交通变化。HorizonWeaver的核心贡献包括数据、模型和训练三方面的互补改进:数据方面构建大规模配对真实/合成数据集;模型方面采用语言引导掩码实现精细编辑;训练方面通过联合损失确保场景一致性和指令忠实性。HorizonWeaver提供可扩展框架,生成255K张图像覆盖13种编辑类别,在L1、CLIP和DINO指标上优于现有方法,用户偏好提升46.4%,BEV分割IoU提升33%。

英文摘要:

Ensuring safety in autonomous driving requires scalable generation of realistic, controllable driving scenes beyond what real-world testing provides. Yet existing instruction guided image editors, trained on object-centric or artistic data, struggle with dense, safety-critical driving layouts. We propose HorizonWeaver, which tackles three fundamental challenges in driving scene editing: (1) multi-level granularity, requiring coherent object- and scene-level edits in dense environments; (2) rich high-level semantics, preserving diverse objects while following detailed instructions; and (3) ubiquitous domain shifts, handling changes in climate, layout, and traffic across unseen environments. The core of HorizonWeaver is a set of complementary contributions across data, model, and training: (1) Data: Large-scale dataset generation, where we build a paired real/synthetic dataset from Boreas, nuScenes, and Argoverse2 to improve generalization; (2) Model: Language-Guided Masks for fine-grained editing, where semantics-enriched masks and prompts enable precise, language-guided edits; and (3) Training: Content preservation and instruction alignment, where joint losses enforce scene consistency and instruction fidelity. Together, HorizonWeaver provides a scalable framework for photorealistic, instruction-driven editing of complex driving scenes, collecting 255K images across 13 editing categories and outperforming prior methods in L1, CLIP, and DINO metrics, achieving +46.4% user preference and improving BEV segmentation IoU by +33%. Project page: https://msoroco.github.io/horizonweaver/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑