CleanStyle: 用于文本到图像风格化任务的即插即用风格条件净化框架
CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylization
AI总结：
CleanStyle通过动态抑制风格嵌入的尾部成分，提升文本到图像风格化任务中提示保真度和视觉质量。
摘要：
在扩散模型中进行风格迁移可以通过注入参考图像的风格来实现可控的视觉生成。然而，最近的基于编码器的方法虽然高效且无需调参，但常常会受到内容泄露的影响，即风格图像中的语义元素不希望地出现在输出中，损害提示保真度和风格一致性。在本工作中，我们引入了CleanStyle，一个即插即用的框架，该框架能够在不重新训练的情况下从风格嵌入中过滤出与内容相关噪声。受经验分析的启发，我们发现这种泄露主要源于风格嵌入的尾部成分，这些成分通过奇异值分解（SVD）被隔离。为了解决这个问题，我们提出了CleanStyleSVD（CS-SVD），它通过时间感知的指数调度动态地抑制尾部成分，在去噪过程中提供干净的、保持风格的条件嵌入。此外，我们还提出了Style-Specific Classifier-Free Guidance（SS-CFG），它重用被抑制的尾部成分来构建具有风格意识的无条件输入。与传统方法使用通用的负嵌入（例如零向量）不同，SS-CFG引入了针对风格特定但提示无关的视觉元素的定向负信号。这使模型能够有效地在生成过程中抑制这些令人分心的模式，从而提高提示保真度并提升整体视觉质量。我们的方法是轻量级的、可解释的，并且可以无缝集成到现有的基于编码器的扩散模型中，而无需重新训练。广泛的实验表明，CleanStyle显著减少了内容泄露，提高了风格化质量，并在广泛的风格参考和提示中改进了提示对齐。
英文摘要：
Style transfer in diffusion models enables controllable visual generation by injecting the style of a reference image. However, recent encoder-based methods, while efficient and tuning-free, often suffer from content leakage, where semantic elements from the style image undesirably appear in the output, impairing prompt fidelity and stylistic consistency. In this work, we introduce CleanStyle, a plug-and-play framework that filters out content-related noise from the style embedding without retraining. Motivated by empirical analysis, we observe that such leakage predominantly stems from the tail components of the style embedding, which are isolated via Singular Value Decomposition (SVD). To address this, we propose CleanStyleSVD (CS-SVD), which dynamically suppresses tail components using a time-aware exponential schedule, providing clean, style-preserving conditional embeddings throughout the denoising process. Furthermore, we present Style-Specific Classifier-Free Guidance (SS-CFG), which reuses the suppressed tail components to construct style-aware unconditional inputs. Unlike conventional methods that use generic negative embeddings (e.g., zero vectors), SS-CFG introduces targeted negative signals that reflect style-specific but prompt-irrelevant visual elements. This enables the model to effectively suppress these distracting patterns during generation, thereby improving prompt fidelity and enhancing the overall visual quality of stylized outputs. Our approach is lightweight, interpretable, and can be seamlessly integrated into existing encoder-based diffusion models without retraining. Extensive experiments demonstrate that CleanStyle substantially reduces content leakage, improves stylization quality and improves prompt alignment across a wide range of style references and prompts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
