h-Edit:基于Doob的h-变换的有效且灵活的扩散式编辑
h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform
AI总结:
本文提出基于Doob的h-变换的扩散图像编辑框架h-Edit,将样本更新分解为重建项和编辑项,实现灵活的免训练复杂编辑,性能优于现有基线。
摘要:
我们通过将基于扩散的图像编辑形式化为逆向时间桥接建模问题,引入了一个理论框架。该方法修改预训练扩散模型的后向过程,以构建一个在时间0收敛到与编辑目标相关的隐式分布的桥。在此框架基础上,我们提出了h-Edit,一种利用Doob的h-变换和Langevin Monte Carlo将中间编辑样本的更新分解为两个组件的新颖编辑方法:一个“重建”项和一个“编辑”项。这种分解提供了灵活性,允许通过现有反演技术计算重建项,并能够组合多个编辑项以处理复杂编辑任务。据我们所知,h-Edit是首个能够同时执行文本引导和基于奖励模型编辑的免训练方法。大量定量和定性实验表明,h-Edit在编辑有效性和忠实度方面优于最先进的基线方法。我们的源代码可在https://github.com/nktoan/h-edit获取。
英文摘要:
We introduce a theoretical framework for diffusion-based image editing by formulating it as a reverse-time bridge modeling problem. This approach modifies the backward process of a pretrained diffusion model to construct a bridge that converges to an implicit distribution associated with the editing target at time 0. Building on this framework, we propose h-Edit, a novel editing method that utilizes Doob's h-transform and Langevin Monte Carlo to decompose the update of an intermediate edited sample into two components: a "reconstruction" term and an "editing" term. This decomposition provides flexibility, allowing the reconstruction term to be computed via existing inversion techniques and enabling the combination of multiple editing terms to handle complex editing tasks. To our knowledge, h-Edit is the first training-free method capable of performing simultaneous text-guided and reward-model-based editing. Extensive experiments, both quantitative and qualitative, show that h-Edit outperforms state-of-the-art baselines in terms of editing effectiveness and faithfulness. Our source code is available at https://github.com/nktoan/h-edit.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。