通过对比学习桥接文本与图像以实现艺术家风格迁移
Bridging Text and Image for Artist Style Transfer via Contrastive Learning
AI总结：
本文提出 CLAST，借助 CLIP、有监督对比训练及基于 adaLN 的状态空间模型，实现无需参考图和在线微调的文本驱动艺术风格迁移。
摘要：
图像风格迁移在过去几年中引起了广泛关注。尽管其效果显著，但它需要额外的风格图像作为参考，这使其灵活性较低且不够方便。使用文本是描述风格最自然的方式。更重要的是，文本可以描述隐含的抽象风格，例如特定艺术家或艺术运动的风格。在本文中，我们提出了用于艺术风格迁移的对比学习方法 Contrastive Learning for Artistic Style Transfer（CLAST），该方法利用先进的图像-文本编码器来控制任意风格迁移。我们引入了一种有监督对比训练策略，以有效地从图像-文本模型（即 CLIP）中提取风格描述，从而使风格化与文本描述保持一致。为此，我们还提出了一种新颖且高效的基于 adaLN 的状态空间模型，用于探索风格与内容的融合。最终，我们实现了文本驱动的图像风格迁移。大量实验表明，我们的方法在艺术风格迁移方面优于最先进的方法。更重要的是，它不需要在线微调，并且能够在 0.03 秒内渲染一张 512×512 的图像。
英文摘要：
Image style transfer has attracted widespread attention in the past few years. Despite its remarkable results, it requires additional style images available as references, making it less flexible and inconvenient. Using text is the most natural way to describe the style. More importantly, text can describe implicit abstract styles, like styles of specific artists or art movements. In this paper, we propose a Contrastive Learning for Artistic Style Transfer (CLAST) that leverages advanced image-text encoders to control arbitrary style transfer. We introduce a supervised contrastive training strategy to effectively extract style descriptions from the image-text model (i.e., CLIP), which aligns stylization with the text description. To this end, we also propose a novel and efficient adaLN based state space models that explore style-content fusion. Finally, we achieve a text-driven image style transfer. Extensive experiments demonstrate that our approach outperforms the state-of-the-art methods in artistic style transfer. More importantly, it does not require online fine-tuning and can render a 512x512 image in 0.03s.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
