扩散模型中的风格注入：一种用于适配大规模扩散模型进行风格迁移的免训练方法
Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer
AI总结：
提出一种基于预训练大规模扩散模型的免训练风格迁移方法，通过替换自注意力层的键值并引入查询保留、温度缩放和AdaIN技术，在保留内容的同时实现高质量风格迁移，超越现有最先进方法。
摘要：
尽管扩散模型具有令人印象深刻的生成能力，但现有的基于扩散模型的风格迁移方法需要推理阶段优化（例如微调或风格的文本反转），这非常耗时，或者无法利用大规模扩散模型的生成能力。为了解决这些问题，我们引入了一种基于预训练大规模扩散模型的全新艺术风格迁移方法，无需任何优化。具体而言，我们操纵自注意力层的特征，其方式类似于交叉注意力机制的工作原理；在生成过程中，将内容的键和值替换为风格图像的键和值。该方法为风格迁移提供了几个理想特性，包括1）通过将相似风格迁移到相似图像块来保留内容，以及2）基于内容与风格图像之间局部纹理（例如边缘）的相似性进行风格迁移。此外，我们引入了查询保留和注意力温度缩放以缓解原始内容被破坏的问题，并引入初始潜在自适应实例归一化（AdaIN）来处理不和谐的颜色（未能迁移风格的颜色）。我们的实验结果表明，我们提出的方法在传统和基于扩散的风格迁移基线上均超越了现有最先进的方法。
英文摘要：
Despite the impressive generative capabilities of diffusion models, existing diffusion model-based style transfer methods require inference-stage optimization (e.g. fine-tuning or textual inversion of style) which is time-consuming, or fails to leverage the generative ability of large-scale diffusion models. To address these issues, we introduce a novel artistic style transfer method based on a pre-trained large-scale diffusion model without any optimization. Specifically, we manipulate the features of self-attention layers as the way the cross-attention mechanism works; in the generation process, substituting the key and value of content with those of style image. This approach provides several desirable characteristics for style transfer including 1) preservation of content by transferring similar styles into similar image patches and 2) transfer of style based on similarity of local texture (e.g. edge) between content and style images. Furthermore, we introduce query preservation and attention temperature scaling to mitigate the issue of disruption of original content, and initial latent Adaptive Instance Normalization (AdaIN) to deal with the disharmonious color (failure to transfer the colors of style). Our experimental results demonstrate that our proposed method surpasses state-of-the-art methods in both conventional and diffusion-based style transfer baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
