EditInfinity：基于二值量化生成模型的图像编辑
EditInfinity: Image Editing with Binary-Quantized Generative Models
AI总结：
本文提出EditInfinity，通过适配二值量化生成模型Infinity，利用其精确中间量化表示改进图像反演，并结合提示校正、风格保持与平滑策略，在PIE-Bench上实现优于扩散基线的文本驱动图像编辑。
摘要：
以可忽略的微调开销将预训练的基于扩散的生成模型适配到文本驱动的图像编辑，已展现出显著潜力。这些方法所遵循的经典适配范式是：首先通过图像反演对给定源图像逆向推断生成轨迹，然后在目标文本提示的引导下沿推断出的轨迹执行图像编辑。然而，图像编辑的性能受到扩散模型在图像反演过程中引入的近似误差的严重限制，这些误差源于中间生成步骤缺乏精确监督。为规避这一问题，我们研究了二值量化生成模型在图像编辑中的参数高效适配，并利用其固有特性——源图像的精确中间量化表示是可获得的，从而为精确图像反演提供更有效的监督。具体而言，我们提出EditInfinity，将二值量化生成模型Infinity适配用于图像编辑。我们提出了一种高效且有效的图像反演机制，该机制融合了文本提示校正和图像风格保持，从而实现精确的图像反演。此外，我们设计了一种整体平滑策略，使EditInfinity能够在图像编辑中保持对源图像的高保真度，并与文本提示实现精确的语义对齐。在PIE-Bench基准上针对“添加”、“更改”和“删除”编辑操作进行的大量实验表明，与最先进的基于扩散的基线方法相比，我们的模型具有更优越的性能。代码可在以下地址获取：https://github.com/yx-chen-ust/EditInfinity。
英文摘要：
Adapting pretrained diffusion-based generative models for text-driven image editing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the generative trajectory inversely for a given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing is heavily limited by the approximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of binary-quantized generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of a source image are attainable, enabling more effective supervision for precise image inversion. Specifically, we propose EditInfinity, which adapts \emph{Infinity}, a binary-quantized generative model, for image editing. We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a holistic smoothing strategy which allows our EditInfinity to perform image editing with high fidelity to source images and precise semantic alignment to the text prompts. Extensive experiments on the PIE-Bench benchmark across `add', `change', and `delete' editing operations, demonstrate the superior performance of our model compared to state-of-the-art diffusion-based baselines. Code available at: https://github.com/yx-chen-ust/EditInfinity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
