UniEdit-I: 基于迭代理解、编辑和验证的无训练图像编辑
UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying
AI总结:
UniEdit-I通过在语义潜在空间中引入迭代理解、编辑和验证循环,实现了无训练的闭环图像编辑,无需微调或架构修改即可达到最先进的性能。
摘要:
尽管统一视觉语言模型旨在协同结合视觉语言模型的高层语义理解和扩散模型的生成保真度,但当前的编辑方法仍然本质上是解耦的和开环的,执行静态的、预定义的转换,没有在语义解释和视觉生成之间进行动态反馈。一个核心限制来自于表征差距:理解通常利用高层、语言对齐的编码器,而生成依赖于低层、像素空间的自编码器,导致特征空间不一致。为弥合这一差距,近期进展如表征自编码器和BLIP3-o倡导直接在预训练语义编码器的高层特征上进行扩散建模。我们发现,在语义潜在空间中进行编辑会修改概念性表示而非像素,确保中间结果在语义上一致且在视觉上合理。基于这一见解,我们提出了UniEdit-I,这是首个无训练、闭环图像编辑框架,通过在统一VLM的语义潜在空间中引入理解-编辑-验证(UEV)循环,将VLM从事后评估者转变为过程中的指挥者,建立了首个以语义驱动、自我纠正的闭环图像编辑流程。在GEdit-Bench上评估时,UniEdit-I在无需任何微调或架构修改的情况下实现了最先进的性能,甚至超过了几个大规模预训练编辑器。
英文摘要:
While Unified Vision-Language Models promise to synergistically combine the high-level semantic understanding of vision-language models with the generative fidelity of diffusion models, current editing methodologies remain fundamentally decoupled and open loop performing static, pre-defined transformations without dynamic feedback between semantic interpretation and visual generation. A central limitation stems from the representation gap: understanding typically leverages high-level, language aligned encoders, whereas generation relies on low level, pixel-space autoencoders, resulting in misaligned feature spaces. To bridge this gap, Recent advances such as Representation Autoencoders and BLIP3-o advocate performing diffusion-based modeling directly in high level features from pretrained semantic encoders. We find editing in the semantic latent space modifies conceptual representations rather than pixels, ensuring intermediates that are both semantically coherent and visually plausible. Building on this insight, We propose UniEdit-I, the first training-free, closed-loop image editing framework that operates entirely within the semantic latent space of a unified VLM by introducing an Understanding-Editing-Verifying (UEV) loop, By transforming the VLM from a posthoc evaluator into an in-process conductor, UniEdit-I establishes the first semantics-driven, self-correcting closed-loop image editing pipeline. Evaluated on GEdit-Bench, UniEdit-I achieves state of the art performance without any fine tuning or architectural modifications, and even surpasses several largescale pretrained editors.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。