StyleDiffusion：用于基于文本编辑的提示词嵌入反演
StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing
AI总结：
本文提出 StyleDiffusion，通过仅优化交叉注意力 value 网络输入并加入注意力正则化，实现结构保持的文本风格编辑，实验证明其优于现有方法。
摘要：
大量研究工作致力于利用预训练扩散模型的惊人能力来进行图像编辑。这些方法要么微调模型，要么在预训练模型的潜空间中反演图像。然而，它们存在两个问题：（1）选定区域的编辑结果不理想，而非选定区域出现意外变化。（2）它们需要仔细进行文本提示词编辑，提示词必须包含输入图像中的所有视觉对象。为解决该问题，我们提出两点改进：（1）仅优化交叉注意力层中 value 线性网络的输入，就足以重建真实图像。（2）我们提出注意力正则化，以在重建和编辑后保留类对象注意力图，从而在不引发显著结构变化的情况下获得准确的风格编辑。我们进一步改进了 P2P 所使用的、用于无分类器引导的无条件分支的编辑技术。在多种图像上进行的大量提示词编辑实验从定性和定量两方面证明，与现有及同期工作相比，我们的方法具有更优越的编辑能力。配套代码见 Stylediffusion：https://github.com/sen-mao/StyleDiffusion。
英文摘要：
A significant research effort is focused on exploiting the amazing capacities of pretrained diffusion models for the editing of images.They either finetune the model, or invert the image in the latent space of the pretrained model. However, they suffer from two problems: (1) Unsatisfying results for selected regions and unexpected changes in non-selected regions.(2) They require careful text prompt editing where the prompt should include all visual objects in the input image.To address this, we propose two improvements: (1) Only optimizing the input of the value linear network in the cross-attention layers is sufficiently powerful to reconstruct a real image. (2) We propose attention regularization to preserve the object-like attention maps after reconstruction and editing, enabling us to obtain accurate style editing without invoking significant structural changes. We further improve the editing technique that is used for the unconditional branch of classifier-free guidance as used by P2P. Extensive experimental prompt-editing results on a variety of images demonstrate qualitatively and quantitatively that our method has superior editing capabilities compared to existing and concurrent works. See our accompanying code in Stylediffusion: \url{https://github.com/sen-mao/StyleDiffusion}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
