发表机构
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对单图像风格迁移中内容保留与风格保真不足的问题，提出Abstract-LoRA方法，通过扩散模型特定U-Net块的轻量级LoRA训练及聚类抽象，实现风格与内容的更好解耦与平衡，提升生成图像的和谐度与保真度。
AI 中文摘要
扩散模型代表了生成建模中最先进的范式之一。借助其发展，越来越多的基于扩散模型的风格迁移方法被提出。然而，在这些方法中，需要至少五到十个风格样本的多图像风格迁移方法往往能取得更令人满意的结果。相比之下，单图像方法常常在内容保留不足或风格保真度不够方面存在问题。这极大地限制了从稀缺艺术品中提取风格，并削弱了其艺术价值。为解决这一问题，我们提出了Abstract-LoRA，一种通过在扩散模型中的特定U-Net块上进行轻量级LoRA训练来推动单图像风格迁移边界的方法。具体来说，我们的工作受B-LoRA启发，B-LoRA是一种通过训练特定U-Net块来实现基本风格-内容解耦的风格迁移方法。然而，它存在一个关键限制：无法捕捉复杂背景。基于B-LoRA，我们的方法对U-Net块进行了更精细的分析，采用额外的U-Net块和基于聚类的风格图像抽象，以更好地解耦并平衡风格与内容。大量实验表明，我们提出的方法不仅生成视觉上更和谐、更令人满意的艺术图像，而且在定量上改善了对最终输出中风格和内容的保留。
英文摘要
Diffusion models represent one of the most advanced paradigms in generative modeling. Leveraging their development, a growing number of style transfer methods based on diffusion models have been proposed. However, among these methods, multi-image style transfer approaches that require at least five to ten style examples tend to achieve more satisfactory results. Single-image methods, by contrast, often struggle with either insufficient content preservation or inadequate style fidelity. This greatly limits style extraction from scarce artworks and undermines their artistic value. To address this, we propose Abstract-LoRA, a method that pushes the boundaries of single-image style transfer through lightweight LoRA training on specific U-Net blocks in diffusion models. Specifically, our work is inspired by B-LoRA, a style transfer method that achieves basic style-content disentanglement by training specific U-Net blocks. However, it suffers from a critical limitation: the inability to capture complex backgrounds. Building upon B-LoRA, our method conducts a more refined analysis of U-Net blocks, employing additional U-Net blocks and clustering-based abstraction of style images to better disentangle and balance style and content. Extensive experiments demonstrate that our proposed method not only generates visually more harmonious and satisfying artistic images but also quantitatively improves the preservation of both style and content in the final outputs.
