To deliver the artistic expression of the target style, recent studies exploit the attention mechanism owing to its ability to map the local patches of the style image to the corresponding patches of the content image. However, because of the low semantic correspondence between arbitrary content and artworks, the attention module repeatedly abuses specific local patches from the style image, resulting in disharmonious and evident repetitive artifacts. To overcome this limitation and accomplish impeccable artistic style transfer, we focus on enhancing the attention mechanism and capturing the rhythm of patterns that organize the style. In this paper, we introduce a novel metric, namely pattern repeatability, that quantifies the repetition of patterns in the style image. Based on the pattern repeatability, we propose Aesthetic Pattern-Aware style transfer Networks (AesPA-Net) that discover the sweet spot of local and global style expressions. In addition, we propose a novel self-supervisory task to encourage the attention mechanism to learn precise and meaningful semantic correspondence. Lastly, we introduce the patch-wise style loss to transfer the elaborate rhythm of local patterns. Through qualitative and quantitative evaluations, we verify the reliability of the proposed pattern repeatability that aligns with human perception, and demonstrate the superiority of the proposed framework.
Recent studies have shown remarkable success in universal style transfer which transfers arbitrary visual styles to content images. However, existing approaches suffer from the aesthetic-unrealistic problem that introduces disharmonious patterns and evident artifacts, making the results easy to spot from real paintings. To address this limitation, we propose AesUST, a novel Aesthetic-enhanced Universal Style Transfer approach that can generate aesthetically more realistic and pleasing results for arbitrary styles. Specifically, our approach introduces an aesthetic discriminator to learn the un…
Ocean University of China（中国海洋大学）；University of Leicester（莱斯特大学）
Textures contain a wealth of image information and are widely used in various fields such as computer graphics and computer vision. With the development of machine learning, the texture synthesis and generation have been greatly improved. As a very common element in everyday life, wallpapers contain a wealth of texture information, making it difficult to annotate with a simple single label. Moreover, wallpaper designers spend significant time to create different styles of wallpaper. For this purpose, this paper proposes to describe wallpaper texture images by using multi-label semantics. Based…
In this paper, we propose a solution to transform a video into a comics. We approach this task using a neural style algorithm based on Generative Adversarial Networks (GANs). Several recent works in the field of Neural Style Transfer showed that producing an image in the style of another image is feasible. In this paper, we build up on these works and extend the existing set of style transfer use cases with a working application of video comixification. To that end, we train an end-to-end solution that transforms input video into a comics in two stages. In the first stage, we propose a state-o…
Department of Computer Science, University of Tübingen（图宾根大学计算机科学系）
Photorealism is a complex concept that cannot easily be formulated mathematically. Deep Photo Style Transfer is an attempt to transfer the style of a reference image to a content image while preserving its photorealism. This is achieved by introducing a constraint that prevents distortions in the content image and by applying the style transfer independently for semantically different parts of the images. In addition, an automated segmentation process is presented that consists of a neural network based segmentation method followed by a semantic grouping step. To further improve the results a …
Many photography websites such as Flickr, 500px, Unsplash, and Adobe Behance are used by amateur and professional photography enthusiasts. Unlike content-based image search, such users of photography websites are not just looking for photos with certain content, but more generally for photos with a certain photographic "aesthetic". In this context, we explore personalized photo recommendation and propose two aesthetic feature extraction methods based on (i) color space and (ii) deep style transfer embeddings. Using a dataset from 500px, we evaluate how these features can be best leveraged by c…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Georgia Institute of Technology（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）；Cornell University（康奈尔大学）
York University（约克大学）；École de technologie supérieure (ÉTS)（高等技术学院（ÉTS））
Image style transfer aims to manipulate the appearance of a source image, or "content" image, to share similar texture and colors of a target "style" image. Ideally, the style transfer manipulation should also preserve the semantic content of the source image. A commonly used approach to assist in transferring styles is based on Gram matrix optimization. One problem of Gram matrix-based optimization is that it does not consider the correlation between colors and their styles. Specifically, certain textures or structures should be associated with specific colors. This is particularly challengin…
Alibaba Group（阿里巴巴集团）；Salesforce Research（赛富时研究院）；NC State University（北卡罗来纳州立大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Winsense Technology Co. Ltd（北京文因互联科技有限公司）
Chinese traditional painting is one of the most historical artworks in the world. It is very popular in Eastern and Southeast Asia due to being aesthetically appealing. Compared with western artistic painting, it is usually more visually abstract and textureless. Recently, neural network based style transfer methods have shown promising and appealing results which are mainly focused on western painting. It remains a challenging problem to preserve abstraction in neural style transfer. In this paper, we present a Neural Abstract Style Transfer method for Chinese traditional painting. It learns …
Training-Free Multi-Style Fusion Through Reference-Based Adaptive Modulation
University of Washington, Seattle, WA 98195, USA（华盛顿大学）；Nanjing University of Posts and Telecommunications, Jiangsu, Nanjing, China（南京邮电大学）；Nanjing Foreign Language School, Jiangsu, Nanjing, China（南京外国语学校）
The aesthetic quality of an image is defined as the measure or appreciation of the beauty of an image. Aesthetics is inherently a subjective property but there are certain factors that influence it such as, the semantic content of the image, the attributes describing the artistic aspect, the photographic setup used for the shot, etc. In this paper we propose a method for the automatic prediction of the aesthetics of an image that is based on the analysis of the semantic content, the artistic style and the composition of the image. The proposed network includes: a pre-trained network for semant…
OmniStyle: Filtering High Quality Style Transfer Data at Scale
Jilin University（吉林大学）；Nanjing University（南京大学）；ByteDance（字节跳动）；Adobe；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China Project page（知识驱动人机智能工程研究中心，中国）
WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models
Luleå University of Technology（吕勒奥理工大学）；National Technical University of Athens（雅典国立技术大学）；Friedrich-Alexander-Universität（弗里德里希-亚历山大大学）；University of West Attica（西阿提卡大学）；University of Ioannina（约阿尼纳大学）
Neural Scene Designer: Self-Styled Semantic Image Manipulation
South China University of Technology（南方科技大学）；Guangdong University of Technology（广东工业大学）；Jimei University（集美大学）；Xiamen Kunlu AI Research Institute（厦门鲲 lun AI 研究院）；Sun Yat-sen University（中山大学）
针对现有图像编辑方法易忽略风格一致性的问题，提出Neural Scene Designer框架，通过双交叉注意力扩散模型和PSRL模块实现语义对齐与风格统一，还构建了综合评估基准验证其有效性。
Neural Style Transfer (NST) is concerned with the artistic stylization of visual media. It can be described as the process of transferring the style of an artistic image onto an ordinary photograph. Recently, a number of studies have considered the enhancement of the depth-preserving capabilities of the NST algorithms to address the undesired effects that occur when the input content images include numerous objects at various depths. Our approach uses a deep residual convolutional network with instance normalization layers that utilizes an advanced depth prediction network to integrate depth p…
Social media images are generally transformed by filtering to obtain aesthetically more pleasing appearances. However, CNNs generally fail to interpret both the image and its filtered version as the same in the visual analysis of social media images. We introduce Instagram Filter Removal Network (IFRNet) to mitigate the effects of image filters for social media analysis applications. To achieve this, we assume any filter applied to an image substantially injects a piece of additional style information to it, and we consider this problem as a reverse style transfer problem. The visual effects o…
Towards In-Context Tone Style Transfer with A Large-Scale Triplet Dataset
VCIP, School of Computer Science, Nankai University（VCIP，计算机科学学院，南开大学）；Nankai International Advanced Research Institute (Shenzhen Futian)（南开国际先进研究院（深圳福田））；OPPO AI Center, OPPO Inc.（OPPO AI中心，OPPO公司）
CreativeSynth: Cross-Art-Attention for Artistic Image Synthesis with Multimodal Diffusion
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；MAIS, Institute of Automation, Chinese Academy of Sciences（自动化所MAIS部）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；University of Chinese Academy of Sciences（中国科学院大学）；ByteDance Inc.（字节跳动公司）；National Cheng Kung University（国立成功大学）
Peking University（北京大学）；Institute of Computer Science and Technology（计算机科学技术研究所）
In this work, we present a new framework for the stylization of text-based binary images. First, our method stylizes the stroke-based geometric shape like text, symbols and icons in the target binary image based on an input style image. Second, the composition of the stylized geometric shape and a background image is explored. To accomplish the task, we propose legibility-preserving structure and texture transfer algorithms, which progressively narrow the visual differences between the binary image and the style image. The stylization is then followed by a context-aware layout design algorithm…
DEAR: Dataset for Evaluating the Aesthetics of Rendering
Color Reproduction and Synthesis Institute（颜色再现与合成研究所）；Moscow Independent Research Institute of Artificial Intelligence（莫斯科独立人工智能研究 institute）；Computational color photography Laboratory（计算色彩摄影实验室）；University of Milano-Bicocca（米兰-比科卡大学）
AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation
Beihang University（北京航空航天大学）；Peking University（北京大学）；Tsinghua University（清华大学）；Huazhong University of Science and Technology（华中科技大学）；Zhongguancun Laboratory（中关村实验室）
FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models
School of Cyberspace Security at Sun Yat-sen University（中山大学网络空间安全学院）；Institute of Software,Chinese Academy of Sciences（中国科学院软件研究所）；Information Technology Service Center of People’s Court（人民法院信息技术服务中心）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）
Shahjalal University of Science and Technology（沙赫贾拉尔科技大学）
In today's world, Neural Style Transfer (NST) has become a trendsetting term. NST combines two pictures, a content picture and a reference image in style (such as the work of a renowned painter) in a way that makes the output image look like an image of the material, but rendered with the form of a reference picture. However, there is no study using the artwork or painting of Bangladeshi painters. Bangladeshi painting has a long history of more than two thousand years and is still being practiced by Bangladeshi painters. This study generates NST stylized image on Bangladeshi paintings and anal…
City University of Hong Kong（香港城市大学）；Meituan-Dianping Group（美团点评集团）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）
Improving the aesthetic quality of images is challenging and eager for the public. To address this problem, most existing algorithms are based on supervised learning methods to learn an automatic photo enhancer for paired data, which consists of low-quality photos and corresponding expert-retouched versions. However, the style and characteristics of photos retouched by experts may not meet the needs or preferences of general users. In this paper, we present an unsupervised image enhancement generative adversarial network (UEGAN), which learns the corresponding image-to-image mapping from a set…
We present a novel approach for disentangling the content of a text image from all aspects of its appearance. The appearance representation we derive can then be applied to new content, for one-shot transfer of the source style to new content. We learn this disentanglement in a self-supervised manner. Our method processes entire word boxes, without requiring segmentation of text from background, per-character processing, or making assumptions on string lengths. We show results in different text domains which were previously handled by specialized methods, e.g., scene text, handwritten text. To…
SITA: Structurally Imperceptible and Transferable Adversarial Attacks for Stylized Image Generation
School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）
Image aesthetics assessment (IAA) is a challenging task due to its highly subjective nature. Most of the current studies rely on large-scale datasets (e.g., AVA and AADB) to learn a general model for all kinds of photography images. However, little light has been shed on measuring the aesthetic quality of artistic images, and the existing datasets only contain relatively few artworks. Such a defect is a great obstacle to the aesthetic assessment of artistic images. To fill the gap in the field of artistic image aesthetics assessment (AIAA), we first introduce a large-scale AIAA dataset: Boldbr…
Fu Foundation School of Engineering and Applied Science（傅 foundation 工程与应用科学学院）；Columbia University（哥伦比亚大学）；University of Michigan（密歇根大学）；Department of Computer Science（计算机科学系）；Lubin School of Business（鲁宾商学院）；Pace University（佩斯大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Few-shot multi-token DreamBooth with LoRa for style-consistent character generation
Institute of Smart Cities (ISC)（智能城市研究所）；Department of Statistics, Computer Science and Mathematics（统计学、计算机科学与数学系）；Public University of Navarre (UPNA)（纳瓦拉公共大学）
Quick Response (QR) code is one of the most worldwide used two-dimensional codes.~Traditional QR codes appear as random collections of black-and-white modules that lack visual semantics and aesthetic elements, which inspires the recent works to beautify the appearances of QR codes. However, these works adopt fixed generation algorithms and therefore can only generate QR codes with a pre-defined style. In this paper, combining the Neural Style Transfer technique, we propose a novel end-to-end method, named ArtCoder, to generate the stylized QR codes that are personalized, diverse, attractive, a…
DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion
Southern University of Science and Technology（南方科技大学）；Beijing Normal University（北京师范大学）；Hebei Guoyan Science and Technology Center（河北国研科技中心）；Wuhan University（武汉大学）；The Australian National University（澳大利亚国立大学）；AI Geeks；Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）
FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion Models
Stanford University（斯坦福大学）；The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；S-Lab, NTU（南洋理工大学S-Lab实验室）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；CPII under InnoHK（InnoHK下属CPII机构）
POSTA: A Go-to Framework for Customized Artistic Poster Generation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；The Hong Kong University of Science and Technology（香港科技大学）
