Neural Abstract Style Transfer for Chinese Traditional Painting
Alibaba Group（阿里巴巴集团）；Salesforce Research（赛富时研究院）；NC State University（北卡罗来纳州立大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Winsense Technology Co. Ltd（北京文因互联科技有限公司）
Chinese traditional painting is one of the most historical artworks in the world. It is very popular in Eastern and Southeast Asia due to being aesthetically appealing. Compared with western artistic painting, it is usually more visually abstract and textureless. Recently, neural network based style transfer methods have shown promising and appealing results which are mainly focused on western painting. It remains a challenging problem to preserve abstraction in neural style transfer. In this paper, we present a Neural Abstract Style Transfer method for Chinese traditional painting. It learns to preserve abstraction and other style jointly end-to-end via a novel MXDoG-guided filter (Modified version of the eXtended Difference-of-Gaussians) and three fully differentiable loss terms. To the best of our knowledge, there is little work study on neural style transfer of Chinese traditional painting. To promote research on this direction, we collect a new dataset with diverse photo-realistic images and Chinese traditional paintings. In experiments, the proposed method shows more appealing stylized results in transferring the style of Chinese traditional painting than state-of-the-art neural style transfer methods.
CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model
Beijing Institute of Technology（北京理工大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Institute of Intelligent Computing Technology, Chinese Academy of Sciences（中国科学院智能计算技术研究所）；Tomorrow Advancing Life（好未来）
A New Chinese Landscape Paintings Generation Model based on Stable Diffusion using DreamBooth
California State University, Long Beach（加州州立大学长滩分校）；Columbia University（哥伦比亚大学）；Northeastern University（东北大学）；No.1 Central Primary School Shanghai Huangpu（上海市黄浦区第一中心小学）；Shanghai Palmim Information Technology Ltd（上海帕米姆信息技术有限公司）
Current GAN-based art generation methods produce unoriginal artwork due to their dependence on conditional input. Here, we propose Sketch-And-Paint GAN (SAPGAN), the first model which generates Chinese landscape paintings from end to end, without conditional input. SAPGAN is composed of two GANs: SketchGAN for generation of edge maps, and PaintGAN for subsequent edge-to-painting translation. Our model is trained on a new dataset of traditional Chinese landscape paintings never before used for generative research. A 242-person Visual Turing Test study reveals that SAPGAN paintings are mistaken …
SRAGAN: Saliency Regularized and Attended Generative Adversarial Network for Chinese Ink-wash Painting Style Transfer
Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）
Collecting a large-scale and well-annotated dataset for image processing has become a common practice in computer vision. However, in the ancient painting area, this task is not practical as the number of paintings is limited and their style is greatly diverse. We, therefore, propose a novel solution for the problems that come with ancient painting processing. This is to use domain transfer to convert ancient paintings to photo-realistic natural images. By doing so, the ancient painting processing problems become natural image processing problems and models trained on natural images can be dir…
DLP-GAN: learning to draw modern Chinese landscape photos with generative adversarial network
School of Computer and Communication, Lanzhou University of Technology（兰州理工大学计算机与通信学院）；School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）
Tongji University（同济大学）；University of Delaware（特拉华大学）
With the advancement of deep learning, artificial intelligence (AI) has made many breakthroughs in recent years and achieved superhuman performance in various tasks such as object detection, reading comprehension, and video games. Generative Modeling, such as various Generative Adversarial Networks (GAN) models, has been applied to generate paintings and music. Research in Natural Language Processing (NLP) also had a leap forward in 2018 since the release of the pre-trained contextual neural language models such as BERT and recently released GPT3. Despite the exciting AI applications aforement…
GANs (Generative adversarial networks) is a new AI technology that can perform deep learning with less training data and has the capability of achieving transformation between two image sets. Using GAN we have carried out a comparison between several art sets with different art style. We have prepared several image sets; a flower photo set (A), an art image set (B1) of Impressionism drawings, an art image set of abstract paintings (B2), an art image set of Chinese figurative paintings, (B3), and an art image set of abstract images (B4) created by Naoko Tosa, one of the authors. Transformation …
PaCaNet: A Study on CycleGAN with Transfer Learning for Diversifying Fused Chinese Painting and Calligraphy
Nanyang Technological University（南洋理工大学）；Seafog AI（海雾人工智能）；San Diego State University（圣地亚哥州立大学）；Heriot-Watt University（赫瑞瓦特大学）；Kyoto University（京都大学）；Rhode Island School of Design（罗德岛设计学院）；Ocean University of China（中国海洋大学）
University of Amsterdam（阿姆斯特丹大学）；Catholic University of Leuven（鲁汶天主教大学）
In this work we propose a new task: artistic visualization of classical Chinese poems, where the goal is to generatepaintings of a certain artistic style for classical Chinese poems. For this purpose, we construct a new dataset called Paint4Poem. Thefirst part of Paint4Poem consists of 301 high-quality poem-painting pairs collected manually from an influential modern Chinese artistFeng Zikai. As its small scale poses challenges for effectively training poem-to-painting generation models, we introduce the secondpart of Paint4Poem, which consists of 3,648 caption-painting pairs collected manuall…
We report the degree of order of twenty-two Jackson Pollock's paintings using \emph{Hausdorff-Besicovitch fractal dimension}. Through the maximum value of each multi-fractal spectrum, the artworks are classify by the year in which they were painted. It has been reported that Pollock's paintings are fractal and it increased on his latest works. However our results show that fractal dimension of the paintings are on a range of fractal dimension with values close to two. We identify this behavior as a fractal-order transition. Based on the study of disorder-order transition in physical systems, w…
The work by Gatys et al. [1] recently showed a neural style algorithm that can produce an image in the style of another image. Some further works introduced various improvements regarding generalization, quality and efficiency, but each of them was mostly focused on styles such as paintings, abstract images or photo-realistic style. In this paper, we present a comparison of how state-of-the-art style transfer methods cope with transferring various comic styles on different images. We select different combinations of Adaptive Instance Normalization [11] and Universal Style Transfer [16] models …
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Style transfer of 3D faces has gained more and more attention. However, previous methods mainly use images of artistic faces for style transfer while ignoring arbitrary style images such as abstract paintings. To solve this problem, we propose a novel method, namely Face-guided Dual Style Transfer (FDST). To begin with, FDST employs a 3D decoupling module to separate facial geometry and texture. Then we propose a style fusion strategy for facial geometry. Subsequently, we design an optimization-based DDSG mechanism for textures that can guide the style transfer by two style images. Besides the…
This extended abstract presents the first stages of a research on in-painting suited for art reconstruction. We introduce M.C Eschers Print Gallery lithography as a use case example. This artwork presents a void on its center and additionally, it follows a challenging mathematical structure that needs to be preserved by the in-painting method. We present our work so far and our future line of research.
Recent studies using deep neural networks have shown remarkable success in style transfer especially for artistic and photo-realistic images. However, the approaches using global feature correlations fail to capture small, intricate textures and maintain correct texture scales of the artworks, and the approaches based on local patches are defective on global effect. In this paper, we present a novel feature pyramid fusion neural network, dubbed GLStyleNet, which sufficiently takes into consideration multi-scale and multi-level pyramid features by best aggregating layers across a VGG network, a…
In recent years, the automatic generation of classical Chinese poetry has made great progress. Besides focusing on improving the quality of the generated poetry, there is a new topic about generating poetry from an image. However, the existing methods for this topic still have the problem of topic drift and semantic inconsistency, and the image-poem pairs dataset is hard to be built when training these models. In this paper, we extract and integrate the Concrete and Abstract information from images to address those issues. We proposed an infilling-based Chinese poetry generation model which ca…
Name Your Style: An Arbitrary Artist-aware Image Style Transfer
Caritas Institute of Higher Education（明爱高等教育学院）；The Hong Kong Polytechnic University（香港理工大学）；École Polytechnique（巴黎综合理工学院）；CNRS（法国国家科学研究中心）；IP Paris（巴黎理工学院）
Can MLLMs Understand the Deep Implication Behind Chinese Images?
Huazhong University of Science and Technology（华中科技大学）；Shenzhen Institute of Advanced Technology, CAS（中国科学院深圳先进技术研究院）；University of Science and Technology of China（中国科学技术大学）；CDUT（成都理工大学）；University of Memphis（孟菲斯大学）；University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；SICAU（四川农业大学）；University of Manchester（曼彻斯特大学）；SWU（西南大学）；UCAS（中国科学院大学）；UMich（密歇根大学）
