End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks
Princeton University（普林斯顿大学）
Alice Xue
备注：
This research is an extension of Alice Xue's senior thesis at Princeton University. The paper will be published in the proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV) 2021 and presented at the conference
Current GAN-based art generation methods produce unoriginal artwork due to their dependence on conditional input. Here, we propose Sketch-And-Paint GAN (SAPGAN), the first model which generates Chinese landscape paintings from end to end, without conditional input. SAPGAN is composed of two GANs: SketchGAN for generation of edge maps, and PaintGAN for subsequent edge-to-painting translation. Our model is trained on a new dataset of traditional Chinese landscape paintings never before used for generative research. A 242-person Visual Turing Test study reveals that SAPGAN paintings are mistaken as human artwork with 55% frequency, significantly outperforming paintings from baseline GANs. Our work lays a groundwork for truly machine-original art generation.
Alibaba Group（阿里巴巴集团）；Salesforce Research（赛富时研究院）；NC State University（北卡罗来纳州立大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Winsense Technology Co. Ltd（北京文因互联科技有限公司）
Chinese traditional painting is one of the most historical artworks in the world. It is very popular in Eastern and Southeast Asia due to being aesthetically appealing. Compared with western artistic painting, it is usually more visually abstract and textureless. Recently, neural network based style transfer methods have shown promising and appealing results which are mainly focused on western painting. It remains a challenging problem to preserve abstraction in neural style transfer. In this paper, we present a Neural Abstract Style Transfer method for Chinese traditional painting. It learns …
CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model
Beijing Institute of Technology（北京理工大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Institute of Intelligent Computing Technology, Chinese Academy of Sciences（中国科学院智能计算技术研究所）；Tomorrow Advancing Life（好未来）
A New Chinese Landscape Paintings Generation Model based on Stable Diffusion using DreamBooth
California State University, Long Beach（加州州立大学长滩分校）；Columbia University（哥伦比亚大学）；Northeastern University（东北大学）；No.1 Central Primary School Shanghai Huangpu（上海市黄浦区第一中心小学）；Shanghai Palmim Information Technology Ltd（上海帕米姆信息技术有限公司）
SRAGAN: Saliency Regularized and Attended Generative Adversarial Network for Chinese Ink-wash Painting Style Transfer
Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）
DLP-GAN: learning to draw modern Chinese landscape photos with generative adversarial network
School of Computer and Communication, Lanzhou University of Technology（兰州理工大学计算机与通信学院）；School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）
Collecting a large-scale and well-annotated dataset for image processing has become a common practice in computer vision. However, in the ancient painting area, this task is not practical as the number of paintings is limited and their style is greatly diverse. We, therefore, propose a novel solution for the problems that come with ancient painting processing. This is to use domain transfer to convert ancient paintings to photo-realistic natural images. By doing so, the ancient painting processing problems become natural image processing problems and models trained on natural images can be dir…
Tongji University（同济大学）；University of Delaware（特拉华大学）
With the advancement of deep learning, artificial intelligence (AI) has made many breakthroughs in recent years and achieved superhuman performance in various tasks such as object detection, reading comprehension, and video games. Generative Modeling, such as various Generative Adversarial Networks (GAN) models, has been applied to generate paintings and music. Research in Natural Language Processing (NLP) also had a leap forward in 2018 since the release of the pre-trained contextual neural language models such as BERT and recently released GPT3. Despite the exciting AI applications aforement…
University of Amsterdam（阿姆斯特丹大学）；Catholic University of Leuven（鲁汶天主教大学）
In this work we propose a new task: artistic visualization of classical Chinese poems, where the goal is to generatepaintings of a certain artistic style for classical Chinese poems. For this purpose, we construct a new dataset called Paint4Poem. Thefirst part of Paint4Poem consists of 301 high-quality poem-painting pairs collected manually from an influential modern Chinese artistFeng Zikai. As its small scale poses challenges for effectively training poem-to-painting generation models, we introduce the secondpart of Paint4Poem, which consists of 3,648 caption-painting pairs collected manuall…
PaCaNet: A Study on CycleGAN with Transfer Learning for Diversifying Fused Chinese Painting and Calligraphy
Nanyang Technological University（南洋理工大学）；Seafog AI（海雾人工智能）；San Diego State University（圣地亚哥州立大学）；Heriot-Watt University（赫瑞瓦特大学）；Kyoto University（京都大学）；Rhode Island School of Design（罗德岛设计学院）；Ocean University of China（中国海洋大学）
nucl.ai Research Laboratory（nucl.ai研究实验室）；Artificial Intelligence in Creative Industries（创意产业人工智能机构）
Convolutional neural networks (CNNs) have proven highly effective at image synthesis and style transfer. For most users, however, using them as tools can be a challenging task due to their unpredictable behavior that goes against common intuitions. This paper introduces a novel concept to augment such generative architectures with semantic annotations, either by manually authoring pixel labels or using existing solutions for semantic segmentation. The result is a content-aware generative algorithm that offers meaningful control over the outcome. Thus, we increase the quality of images generate…
Artistic style transfer is an image synthesis problem where the content of an image is reproduced with the style of another. Recent works show that a visually appealing style transfer can be achieved by using the hidden activations of a pretrained convolutional neural network. However, existing methods either apply (i) an optimization procedure that works for any style image but is very expensive, or (ii) an efficient feedforward network that only allows a limited number of trained styles. In this work we propose a simpler optimization objective based on local matching that combines the conten…
This article is about the cognitive science of visual art. Artists create physical artifacts (such as sculptures or paintings) which depict people, objects, and events. These depictions are usually stylized rather than photo-realistic. How is it that humans are able to understand and create stylized representations? Does this ability depend on general cognitive capacities or an evolutionary adaptation for art? What role is played by learning and culture? Machine Learning can shed light on these questions. It's possible to train convolutional neural networks (CNNs) to recognize objects without …
Recent studies using deep neural networks have shown remarkable success in style transfer especially for artistic and photo-realistic images. However, the approaches using global feature correlations fail to capture small, intricate textures and maintain correct texture scales of the artworks, and the approaches based on local patches are defective on global effect. In this paper, we present a novel feature pyramid fusion neural network, dubbed GLStyleNet, which sufficiently takes into consideration multi-scale and multi-level pyramid features by best aggregating layers across a VGG network, a…
Nottingham Trent University（诺丁汉特伦特大学）；Department of Computer Science（计算机科学系）；Computational Intelligence and Applications Research Group (CIA)（计算智能与应用研究组（CIA））；Nottingham, United Kingdom（英国诺丁汉（注：此处为地址信息，非机构，按要求应忽略，但原文中该部分与机构列表相邻，需确认是否提取。根据提取范围，地址不属于机构，故最终仅保留大学、学院、系、研究组等机构名称。经再次核对，正确机构为诺丁汉特伦特大学、计算机科学系、计算智能与应用研究组（CIA）。其中“Department of Computer Science”属于系，“Computational Intelligence and Applications Research Group (CIA)”属于研究组，均符合机构粒度要求。）}]} 修正后：{）
Much of the state-of-the-art in image synthesis inspired by real artwork are either entirely generative by filtered random noise or inspired by the transfer of style. This work explores the application of image inpainting to continue famous artworks and produce generative art with a Conditional GAN. During the training stage of the process, the borders of images are cropped, leaving only the centre. An inpainting GAN is then tasked with learning to reconstruct the original image from the centre crop by way of minimising both adversarial and absolute difference losses, which are analysed by bot…
PATCH: a deep learning method to assess heterogeneity of artistic practice in historical paintings
Case Western Reserve University（凯斯西储大学）；Department of Physics（物理系）；Department of Art History and Art（艺术史与艺术系）；The Frick Pittsburgh Museums and Gardens（弗里克匹兹堡博物馆和花园）；Department of Computer and Data Sciences（计算机与数据科学系）；MORE Center（MORE中心）；Cleveland Institute of Art（克利夫兰艺术学院）；Factum Foundation（Factum基金会）
Conventional methods for the image-text generation tasks mainly tackle the naturally bidirectional generation tasks separately, focusing on designing task-specific frameworks to improve the quality and fidelity of the generated samples. Recently, Vision-Language Pre-training models have greatly improved the performance of the image-to-text generation tasks, but large-scale pre-training models for text-to-image synthesis task are still under-developed. In this paper, we propose ERNIE-ViLG, a unified generative pre-training framework for bidirectional image-text generation with transformer model…
