Guide-and-Rescale：用于有效免调谐真实图像编辑的自引导机制
Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing
AI总结：
本文提出一种基于自引导机制的免调谐真实图像编辑方法，通过引入保持布局的能量函数和噪声重缩放机制，在保留原图结构的同时实现快速高质量的编辑。
摘要：
尽管大规模文本到图像生成模型最近取得了进展，但使用这些模型操纵真实图像仍然是一个具有挑战性的问题。现有编辑方法的主要局限性在于，它们要么在广泛的图像编辑中无法保持一致的质量，要么需要进行耗时的超参数调优或对扩散模型进行微调，以保留输入图像的特定外观。我们提出了一种基于通过引导机制修改的扩散采样过程的新方法。在本工作中，我们探索了自引导技术，以保留输入图像的整体结构及其不应被编辑的局部区域外观。具体而言，我们明确引入了保持布局的能量函数，旨在保存源图像的局部和全局结构。此外，我们提出了一种噪声重缩放机制，通过在生成过程中平衡无分类器引导和我们提出的引导器的范数来保留噪声分布。这种引导方法不需要对扩散模型进行微调，也不需要精确的求逆过程。因此，所提出的方法提供了一种快速且高质量的编辑机制。在我们的实验中，通过人类评估和定量分析表明，所提出的方法能够产生人类更偏好的所需编辑，并且在编辑质量和保留原始图像之间取得了更好的权衡。我们的代码可在 https://github.com/MACderRu/Guide-and-Rescale 获取。
英文摘要：
Despite recent advances in large-scale text-to-image generative models, manipulating real images with these models remains a challenging problem. The main limitations of existing editing methods are that they either fail to perform with consistent quality on a wide range of image edits or require time-consuming hyperparameter tuning or fine-tuning of the diffusion model to preserve the image-specific appearance of the input image. We propose a novel approach that is built upon a modified diffusion sampling process via the guidance mechanism. In this work, we explore the self-guidance technique to preserve the overall structure of the input image and its local regions appearance that should not be edited. In particular, we explicitly introduce layout-preserving energy functions that are aimed to save local and global structures of the source image. Additionally, we propose a noise rescaling mechanism that allows to preserve noise distribution by balancing the norms of classifier-free guidance and our proposed guiders during generation. Such a guiding approach does not require fine-tuning the diffusion model and exact inversion process. As a result, the proposed method provides a fast and high-quality editing mechanism. In our experiments, we show through human evaluation and quantitative analysis that the proposed method allows to produce desired editing which is more preferable by humans and also achieves a better trade-off between editing quality and preservation of the original image. Our code is available at https://github.com/MACderRu/Guide-and-Rescale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
