arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.08677cs.CV

OmniPaint:通过解耦插入-移除修复掌握面向对象的编辑

OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting

University of Rochester(罗切斯特大学);Adobe Research(奥多比研究院)

Yongsheng Yu, Ziyun Zeng, Haitian Zheng, Jiebo Luo

AI总结:

本文提出统一框架OmniPaint,将对象移除与插入建模为相互依赖的过程,结合扩散先验、渐进式训练和CycleFlow无配对细化,并引入CFD无参考评估指标,实现高保真且场景一致的图像编辑。

摘要:

基于扩散的生成模型已经彻底改变了面向对象的图像编辑,然而它们在现实对象移除和插入中的部署仍然受到诸如物理效应复杂相互作用和成对训练数据不足等挑战的阻碍。在这项工作中,我们引入了OmniPaint,一个统一框架,将对象移除和插入重新概念化为相互依赖的过程,而非孤立的任务。OmniPaint利用预训练的扩散先验以及一个渐进式训练流程,该流程包括初始成对样本优化和随后通过CycleFlow进行的大规模无配对细化,实现了精确的前景消除和无缝对象插入,同时忠实地保留场景几何和固有属性。此外,我们新颖的CFD指标提供了一种稳健的、无参考的评估方法,用于衡量上下文一致性和对象幻觉,为高保真图像编辑建立了新的基准。项目页面:https://yeates.github.io/OmniPaint-Page/

英文摘要:

Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data. In this work, we introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes rather than isolated tasks. Leveraging a pre-trained diffusion prior along with a progressive training pipeline comprising initial paired sample optimization and subsequent large-scale unpaired refinement via CycleFlow, OmniPaint achieves precise foreground elimination and seamless object insertion while faithfully preserving scene geometry and intrinsic properties. Furthermore, our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination, establishing a new benchmark for high-fidelity image editing. Project page: https://yeates.github.io/OmniPaint-Page/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑