arXiv 2503.142752025-12-12cs.CV

无需调优的色彩-纹理解耦用于风格化图像生成

Free-Lunch Color-Texture Disentanglement for Stylized Image Generation

Harbin Institute of Technology, China（哈尔滨工业大学）；VCIP, CS, Nankai University, China（南开大学）；Computer Vision Center, Spain（西班牙计算机视觉中心）；Universitat Autònoma de Barcelona, Spain（巴塞罗那自治大学）；Program of Computer Science, City University of Hong Kong (Dongguan), China（香港城市大学（东莞）计算机科学系）；City University of Hong Kong, HK SAR, China（香港城市大学）

Jiang Qin, Senmao Li, Alexandra Gomez-Villa, Shiqi Yang, Yaxing Wang, Kai Wang, Joost van de Weijer

备注：: Accepted by NeurIPS2025. Code is available at https://deepffff.github.io/sadis.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出无需调优的SADis方法，通过分离颜色-纹理嵌入提升风格化图像生成的精度与定制能力。

摘要：

最近在文本到图像（T2I）扩散模型中的进展已经改变了图像生成，使仅使用少量风格参考图像就能在风格化生成中取得显著进展。然而，当前基于扩散的方法在细粒度风格定制上面临挑战，因为控制多种风格属性（如颜色和纹理）存在困难。本文介绍了首个无需调优的方法，以实现风格化T2I生成中的免费午餐色彩-纹理解耦，解决Disentangled Stylized Image Generation（DisIG）问题中对独立控制风格元素的需求。我们的方法利用CLIP图像嵌入空间中的图像提示加法性属性，开发出从单个颜色和纹理参考图像中分离和提取颜色-纹理嵌入（CTE）的技术。为了确保生成图像的颜色调色板与颜色参考图像紧密一致，我们应用了白化和着色变换以增强颜色一致性。此外，为了防止由于扩散训练中固有的信号泄漏偏差导致的纹理损失，我们引入了一个噪声项，在正则化白化和着色变换（RegWCT）过程中保持纹理保真度。通过这些方法，我们的风格属性解耦方法（SADis）为风格化图像生成提供了更精确和可定制的解决方案。在WikiArt和StyleDrop数据集上的实验表明，SADis在DisIG任务中在定性和定量上均优于最先进的风格化方法。代码已发布在https://deepffff.github.io/sadis.github.io/。

英文摘要：

Recent advances in Text-to-Image (T2I) diffusion models have transformed image generation, enabling significant progress in stylized generation using only a few style reference images. However, current diffusion-based methods struggle with fine-grained style customization due to challenges in controlling multiple style attributes, such as color and texture. This paper introduces the first tuning-free approach to achieve free-lunch color-texture disentanglement in stylized T2I generation, addressing the need for independently controlled style elements for the Disentangled Stylized Image Generation (DisIG) problem. Our approach leverages the Image-Prompt Additivity property in the CLIP image embedding space to develop techniques for separating and extracting Color-Texture Embeddings (CTE) from individual color and texture reference images. To ensure that the color palette of the generated image aligns closely with the color reference, we apply a whitening and coloring transformation to enhance color consistency. Additionally, to prevent texture loss due to the signal-leak bias inherent in diffusion training, we introduce a noise term that preserves textural fidelity during the Regularized Whitening and Coloring Transformation (RegWCT). Through these methods, our Style Attributes Disentanglement approach (SADis) delivers a more precise and customizable solution for stylized image generation. Experiments on images from the WikiArt and StyleDrop datasets demonstrate that, both qualitatively and quantitatively, SADis surpasses state-of-the-art stylization methods in the DisIG task.Code is released at https://deepffff.github.io/sadis.github.io/.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。