U-StyDiT：使用扩散Transformer的超高质量艺术风格迁移
U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformers
AI总结：
本文提出基于扩散Transformer的U-StyDiT方法，通过多视角风格调制器和StyDiT Block实现内容-风格解耦，并构建Aes4M数据集，以生成超高质量艺术风格化图像。
摘要：
超高质量艺术风格迁移是指利用从风格图像中学习到的风格信息，对超高质量内容图像进行重新绘制。现有的艺术风格迁移方法可分为基于风格重建和基于内容-风格解耦的风格迁移方法。尽管这些方法能够生成一些艺术风格化图像，但它们仍然存在明显的伪影和不协调的图案，这阻碍了它们生成超高质量艺术风格化图像的能力。为了解决这些问题，我们提出了一种新颖的艺术图像风格迁移方法U-StyDiT，该方法基于Transformer扩散（DiT）构建，并学习内容-风格解耦，从而生成超高质量的艺术风格化图像。具体来说，我们首先设计了一个多视角风格调制器（MSM），从局部和全局视角学习风格图像中的风格信息，并以此条件化U-StyDiT，使其利用学习到的风格信息生成风格化图像。然后，我们引入StyDiT Block，从风格图像中同时学习内容和风格条件。此外，我们提出了一个超高质量艺术图像数据集Aes4M，该数据集包含10个类别，每个类别包含400,000张风格图像。该数据集有效解决了现有风格迁移方法由于数据集规模和图像质量限制而无法生成高质量艺术风格化图像的问题。最后，大量的定性和定量实验验证了我们的U-StyDiT与最先进的艺术风格迁移方法相比，能够生成更高质量的风格化图像。据我们所知，我们提出的方法是首个利用基于Transformer的扩散来解决超高质量风格化图像生成问题的方法。
英文摘要：
Ultra-high quality artistic style transfer refers to repainting an ultra-high quality content image using the style information learned from the style image. Existing artistic style transfer methods can be categorized into style reconstruction-based and content-style disentanglement-based style transfer approaches. Although these methods can generate some artistic stylized images, they still exhibit obvious artifacts and disharmonious patterns, which hinder their ability to produce ultra-high quality artistic stylized images. To address these issues, we propose a novel artistic image style transfer method, U-StyDiT, which is built on transformer-based diffusion (DiT) and learns content-style disentanglement, generating ultra-high quality artistic stylized images. Specifically, we first design a Multi-view Style Modulator (MSM) to learn style information from a style image from local and global perspectives, conditioning U-StyDiT to generate stylized images with the learned style information. Then, we introduce a StyDiT Block to learn content and style conditions simultaneously from a style image. Additionally, we propose an ultra-high quality artistic image dataset, Aes4M, comprising 10 categories, each containing 400,000 style images. This dataset effectively solves the problem that the existing style transfer methods cannot produce high-quality artistic stylized images due to the size of the dataset and the quality of the images in the dataset. Finally, the extensive qualitative and quantitative experiments validate that our U-StyDiT can create higher quality stylized images compared to state-of-the-art artistic style transfer methods. To our knowledge, our proposed method is the first to address the generation of ultra-high quality stylized images using transformer-based diffusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
