用于扩散模型中概念的细粒度控制、编辑和擦除的Prompt Sliders
Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Models
AI总结：
针对扩散模型中概念控制依赖LoRA导致推理慢且跨模型泛化差的问题，提出Prompt Sliders方法，通过文本嵌入学习概念，实现细粒度控制、编辑与擦除，速度提升30%且存储仅需3KB。
摘要：
扩散模型近期在图像合成和编辑方面超越了GANs，提供了更优的图像质量和多样性。然而，对生成图像中的属性实现精确控制仍然是一个挑战。Concept Sliders提出了一种通过学习概念（属性/对象）来实现细粒度图像控制和编辑的方法。然而，由于需要加载和卸载用于学习概念的Low-Rank Adapters (LoRAs)，这种方法增加了参数并延长了推理时间。这些适配器是特定于模型的，并且需要针对不同的架构（例如Stable Diffusion (SD) v1.5和SD-XL）重新训练。在本文中，我们提出了一种直接的文本反演方法，通过文本嵌入来学习概念，这些嵌入可泛化至共享相同文本编码器的模型，包括不同版本的SD模型。我们将我们的方法称为Prompt Sliders。除了学习新概念外，我们还展示了Prompt Sliders可用于擦除不良概念，如艺术风格或成人内容。我们的方法比使用LoRAs快30%，因为它消除了加载和卸载适配器的需要，并且除了目标概念文本嵌入外不引入额外参数。每个概念嵌入仅需3KB的存储空间，而每个LoRA适配器需要8922KB或更多，这使得我们的方法在计算上更加高效。项目页面：https://deepaksridhar.github.io/promptsliders.github.io/
英文摘要：
Diffusion models have recently surpassed GANs in image synthesis and editing, offering superior image quality and diversity. However, achieving precise control over attributes in generated images remains a challenge. Concept Sliders introduced a method for fine-grained image control and editing by learning concepts (attributes/objects). However, this approach adds parameters and increases inference time due to the loading and unloading of Low-Rank Adapters (LoRAs) used for learning concepts. These adapters are model-specific and require retraining for different architectures, such as Stable Diffusion (SD) v1.5 and SD-XL. In this paper, we propose a straightforward textual inversion method to learn concepts through text embeddings, which are generalizable across models that share the same text encoder, including different versions of the SD model. We refer to our method as Prompt Sliders. Besides learning new concepts, we also show that Prompt Sliders can be used to erase undesirable concepts such as artistic styles or mature content. Our method is 30% faster than using LoRAs because it eliminates the need to load and unload adapters and introduces no additional parameters aside from the target concept text embedding. Each concept embedding only requires 3KB of storage compared to the 8922KB or more required for each LoRA adapter, making our approach more computationally efficient. Project Page: https://deepaksridhar.github.io/promptsliders.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
