Paintings and Drawings Aesthetics Assessment with Rich Attributes for Various Artistic Categories
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Central Academy of Fine Arts（中央美术学院）；Beijing University of Technology（北京工业大学）；University of Science and Technology of China（中国科学技术大学）；Beijing Institute of Fashion Technology（北京服装学院）
Xin Jin, Qianqian Qiao, Yi Lu, Shan Gao, Heng Huang, Guangdong Li
Image aesthetic evaluation is a highly prominent research domain in the field of computer vision. In recent years, there has been a proliferation of datasets and corresponding evaluation methodologies for assessing the aesthetic quality of photographic works, leading to the establishment of a relatively mature research environment. However, in contrast to the extensive research in photographic aesthetics, the field of aesthetic evaluation for paintings and Drawings has seen limited attention until the introduction of the BAID dataset in March 2023. This dataset solely comprises overall scores for high-quality artistic images. Our research marks the pioneering introduction of a multi-attribute, multi-category dataset specifically tailored to the field of painting: Aesthetics of Paintings and Drawings Dataset (APDD). The construction of APDD received active participation from 28 professional artists worldwide, along with dozens of students specializing in the field of art. This dataset encompasses 24 distinct artistic categories and 10 different aesthetic attributes. Each image in APDD has been evaluated by six professionally trained experts in the field of art, including assessments for both total aesthetic scores and aesthetic attribute scores. The final APDD dataset comprises a total of 4985 images, with an annotation count exceeding 31100 entries. Concurrently, we propose an innovative approach: Art Assessment Network for Specific Painting Styles (AANSPS), designed for the assessment of aesthetic attributes in mixed-attribute art datasets. Through this research, our goal is to catalyze advancements in the field of aesthetic evaluation for paintings and drawings, while enriching the available resources and methodologies for its further development and application.
School of Computer Science and Technology（计算机科学与技术学院）；East China Normal University（东华师范大学）；School of Design（设计学院）
Artistic image assessment has become a prominent research area in computer vision. In recent years, the field has witnessed a proliferation of datasets and methods designed to evaluate the aesthetic quality of paintings. However, most existing approaches focus solely on static final images, overlooking the dynamic and multi-stage nature of the artistic painting process. To address this gap, we propose a novel framework for human-aligned assessment of painting processes. Specifically, we introduce the Painting Process Assessment Dataset (PPAD), the first large-scale dataset comprising real and …
Image aesthetics assessment (IAA) is a challenging task due to its highly subjective nature. Most of the current studies rely on large-scale datasets (e.g., AVA and AADB) to learn a general model for all kinds of photography images. However, little light has been shed on measuring the aesthetic quality of artistic images, and the existing datasets only contain relatively few artworks. Such a defect is a great obstacle to the aesthetic assessment of artistic images. To fill the gap in the field of artistic image aesthetics assessment (AIAA), we first introduce a large-scale AIAA dataset: Boldbr…
APDDv2: Aesthetics of Paintings and Drawings Dataset with Artist Labeled Scores and Comments
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Central Academy of Fine Arts（中央美术学院）；University of Science and Technology of China（中国科学技术大学）；Beijing University of Technology（北京工业大学）
Friedrich Schiller University（弗里德里希·席勒大学）；University of Jena School of Medicine（耶拿大学医学院）；Jena University Hospital（耶拿大学医院）
Computational aesthetics is an emerging field of research which has attracted different research groups in the last few years. In this field, one of the main approaches to evaluate the aesthetic quality of paintings and photographs is a feature-based approach. Among the different features proposed to reach this goal, color plays an import role. In this paper, we introduce a novel dataset that consists of paintings of Western provenance from 36 well-known painters from the 15th to the 20th century. As a first step and to assess this dataset, using a classifier, we investigate the correlation be…
Automatic art analysis has been mostly focused on classifying artworks into different artistic styles. However, understanding an artistic representation involves more complex processes, such as identifying the elements in the scene or recognizing author influences. We present SemArt, a multi-modal dataset for semantic art understanding. SemArt is a collection of fine-art painting images in which each image is associated to a number of attributes and a textual artistic comment, such as those that appear in art catalogues or museum collections. To evaluate semantic art understanding, we envisage…
Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）
Drawing a beautiful painting is a dream of many people since childhood. In this paper, we propose a novel scheme, Line Artist, to synthesize artistic style paintings with freehand sketch images, leveraging the power of deep learning and advanced algorithms. Our scheme includes three models. The Sketch Image Extraction (SIE) model is applied to generate the training data. It includes smoothing reality images and pencil sketch extraction. The Detailed Image Synthesis (DIS) model trains a conditional generative adversarial network to generate detailed real-world information. The Adaptively Weight…
The diversity of painting styles represents a rich visual vocabulary for the construction of an image. The degree to which one may learn and parsimoniously capture this visual vocabulary measures our understanding of the higher level features of paintings, if not images in general. In this work we investigate the construction of a single, scalable deep network that can parsimoniously capture the artistic style of a diversity of paintings. We demonstrate that such a network generalizes across a diversity of artistic styles by reducing a painting to a point in an embedding space. Importantly, th…
Google Brain（谷歌大脑）；MILA, Université de Montréal（蒙特利尔大学MILA）
In this paper, we present a method which combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks to allow real-time stylization using any content/style image pair. We build upon recent work leveraging conditional instance normalization for multi-style transfer networks by learning to predict the conditional instance normalization parameters directly from a style image. The model is successfully trained on a corpus of roughly 80,000 paintings and is able to generalize to paintings previously unobserved. We demonstrate that the learned em…
University of Aveiro（阿威罗大学）；University of Helsinki（赫尔辛基大学）
The artistic community is increasingly relying on automatic computational analysis for authentication and classification of artistic paintings. In this paper, we identify hidden patterns and relationships present in artistic paintings by analysing their complexity, a measure that quantifies the sum of characteristics of an object. Specifically, we apply Normalized Compression (NC) and the Block Decomposition Method (BDM) to a dataset of 4,266 paintings from 91 authors and examine the potential of these information-based measures as descriptors of artistic paintings. Both measures consistently …
This note presents an extension to the neural artistic style transfer algorithm (Gatys et al.). The original algorithm transforms an image to have the style of another given image. For example, a photograph can be transformed to have the style of a famous painting. Here we address a potential shortcoming of the original method: the algorithm transfers the colors of the original painting, which can alter the appearance of the scene in undesirable ways. We describe simple linear methods for transferring style while preserving colors.
University of Amsterdam（阿姆斯特丹大学）；Catholic University of Leuven（鲁汶天主教大学）
In this work we propose a new task: artistic visualization of classical Chinese poems, where the goal is to generatepaintings of a certain artistic style for classical Chinese poems. For this purpose, we construct a new dataset called Paint4Poem. Thefirst part of Paint4Poem consists of 301 high-quality poem-painting pairs collected manually from an influential modern Chinese artistFeng Zikai. As its small scale poses challenges for effectively training poem-to-painting generation models, we introduce the secondpart of Paint4Poem, which consists of 3,648 caption-painting pairs collected manuall…
Recent studies have shown remarkable success in universal style transfer which transfers arbitrary visual styles to content images. However, existing approaches suffer from the aesthetic-unrealistic problem that introduces disharmonious patterns and evident artifacts, making the results easy to spot from real paintings. To address this limitation, we propose AesUST, a novel Aesthetic-enhanced Universal Style Transfer approach that can generate aesthetically more realistic and pleasing results for arbitrary styles. Specifically, our approach introduces an aesthetic discriminator to learn the un…
Institute for Datability Science（数据可持续性科学研究所）；Osaka University（大阪大学）
In computer vision, visual arts are often studied from a purely aesthetics perspective, mostly by analysing the visual appearance of an artistic reproduction to infer its style, its author, or its representative features. In this work, however, we explore art from both a visual and a language perspective. Our aim is to bridge the gap between the visual appearance of an artwork and its underlying meaning, by jointly analysing its aesthetics and its semantics. We introduce the use of multi-modal techniques in the field of automatic art analysis by 1) collecting a multi-modal dataset with fine-ar…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of AI, UCAS（中国科学院大学人工智能学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Kuaishou Technology（快手科技）
The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes including semantic elements, object shapes, etc. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. The pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality, but it often requires extensive textual descriptions to accurately portray attributes of a particular painting. We believe that the uniqueness of an …
To facilitate computer analysis of visual art, in the form of paintings, we introduce Pandora (Paintings Dataset for Recognizing the Art movement) database, a collection of digitized paintings labelled with respect to the artistic movement. Noting that the set of databases available as benchmarks for evaluation is highly reduced and most existing ones are limited in variability and number of images, we propose a novel large scale dataset of digital paintings. The database consists of more than 7700 images from 12 art movements. Each genre is illustrated by a number of images varying from 250 t…
An Order-Complexity Aesthetic Assessment Model for Aesthetic-aware Music Recommendation
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Tianjin Conservatory of Music（天津音乐学院）；Qingdao Academy of Intelligent Industries（青岛智能产业技术研究院）；Tongji University（同济大学）；Sun Yat-sen University（中山大学）
The aesthetic quality of an image is defined as the measure or appreciation of the beauty of an image. Aesthetics is inherently a subjective property but there are certain factors that influence it such as, the semantic content of the image, the attributes describing the artistic aspect, the photographic setup used for the shot, etc. In this paper we propose a method for the automatic prediction of the aesthetics of an image that is based on the analysis of the semantic content, the artistic style and the composition of the image. The proposed network includes: a pre-trained network for semant…
The artistic style of a painting is a subtle aesthetic judgment used by art historians for grouping and classifying artwork. The recently introduced `neural-style' algorithm substantially succeeds in merging the perceived artistic style of one image or set of images with the perceived content of another. In light of this and other recent developments in image analysis via convolutional neural networks, we investigate the effectiveness of a `neural-style' representation for classifying the artistic style of paintings.
The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Tel-Aviv University（特拉维夫大学）；Reichman University（赖希曼大学）
Semantic segmentation is a difficult task even when trained in a supervised manner on photographs. In this paper, we tackle the problem of semantic segmentation of artistic paintings, an even more challenging task because of a much larger diversity in colors, textures, and shapes and because there are no ground truth annotations available for segmentation. We propose an unsupervised method for semantic segmentation of paintings using domain adaptation. Our approach creates a training set of pseudo-paintings in specific artistic styles by using style-transfer on the PASCAL VOC 2012 dataset, and…
Rutgers, The State University of New Jersey（罗格斯，新泽西州立大学）
Considering the huge amount of art pieces that exist, there is valuable information to be discovered. Examining a painting, an expert can determine its style, genre, and the time period that the painting belongs. One important task for art historians is to find influences and connections between artists. Is influence a task that a computer can measure? The contribution of this paper is in exploring the problem of computer-automated suggestion of influences between artists, a problem that was not addressed before in a general setting. We first present a comparative study of different classifica…
Converting text descriptions into images using Generative Adversarial Networks has become a popular research area. Visually appealing images have been generated successfully in recent years. Inspired by these studies, we investigated the generation of artistic images on a large variance dataset. This dataset includes images with variations, for example, in shape, color, and content. These variations in images provide originality which is an important factor for artistic essence. One major characteristic of our work is that we used keywords as image descriptions, instead of sentences. As the ne…
PATCH: a deep learning method to assess heterogeneity of artistic practice in historical paintings
Case Western Reserve University（凯斯西储大学）；Department of Physics（物理系）；Department of Art History and Art（艺术史与艺术系）；The Frick Pittsburgh Museums and Gardens（弗里克匹兹堡博物馆和花园）；Department of Computer and Data Sciences（计算机与数据科学系）；MORE Center（MORE中心）；Cleveland Institute of Art（克利夫兰艺术学院）；Factum Foundation（Factum基金会）
ARTxAI: Explainable Artificial Intelligence Curates Deep Representation Learning for Artistic Images using Fuzzy Techniques
Universidad Publica de Navarra（纳瓦拉公立大学）；University of Granada（格拉纳达大学）；Andalusian Research Institute “Data Science and Computational Intelligence”（安达卢西亚“数据科学与计算智能”研究所）；University of Essex（埃塞克斯大学）；University of Jaen（哈恩大学）
The University of Adelaide（阿德莱德大学）；School of Computer Science, The University of Adelaide（阿德莱德大学计算机学院）；Optimisation and Logistics, School of Computer Science, The University of Adelaide（阿德莱德大学计算机学院优化与物流研究组（注：因“研究组”非机构级别，按要求取到学院/大学级别，此处主要机构为The University of Adelaide，故优先提取））
We present a study demonstrating how random walk algorithms can be used for evolutionary image transition. We design different mutation operators based on uniform and biased random walks and study how their combination with a baseline mutation operator can lead to interesting image transition processes in terms of visual effects and artistic features. Using feature-based analysis we investigate the evolutionary image transition behaviour with respect to different features and evaluate the images constructed during the image transition process. Afterwards, we investigate how modifications of ou…
In the past, manually re-drawing an image in a certain artistic style required a professional artist and a long time. Doing this for a video sequence single-handed was beyond imagination. Nowadays computers provide new possibilities. We present an approach that transfers the style from one image (for example, a painting) to a whole video sequence. We make use of recent advances in style transfer in still images and propose new initializations and loss functions applicable to videos. This allows us to generate consistent and stable stylized video sequences, even in cases with large motion and s…
This paper proposes a framework for computational modeling of artistic painting algorithms, inspired by human creative practices. Based on examples from expert artists and from the author's own experience, the paper argues that creative processes often involve two important components: vague, high-level goals (e.g., "make a good painting"), and exploratory processes for discovering new ideas. This paper then sketches out possible computational mechanisms for imitating those elements of the painting process, including underspecified loss functions and iterative painting procedures with explicit…
Facial landmark detection plays an important role for the similarity analysis in artworks to compare portraits of the same or similar artists. With facial landmarks, portraits of different genres, such as paintings and prints, can be automatically aligned using control-point-based image registration. We propose a deep-learning-based method for facial landmark detection in high-resolution images of paintings and prints. It divides the task into a global network for coarse landmark prediction and multiple region networks for precise landmark refinement in regions of the eyes, nose, and mouth tha…
Indian Institute of Information Technology Sri City（斯里城印度信息技术学院）
This paper proposes a technique to assess the aesthetic quality of photographs. The goal of the study is to predict whether a given photograph is captured by professional photographers, or by common people, based on a measurement of artistic quality of the photograph. We propose a Multi-Layer-Perceptron based system to analyze some low, mid and high level image features and find their effectiveness to measure artistic quality of the image and produce a measurement of the artistic quality of the image on a scale of 10. We validate the proposed system on a large dataset, containing images downlo…
The artistic style of a painting is a rich descriptor that reveals both visual and deep intrinsic knowledge about how an artist uniquely portrays and expresses their creative vision. Accurate categorization of paintings across different artistic movements and styles is critical for large-scale indexing of art databases. However, the automatic extraction and recognition of these highly dense artistic features has received little to no attention in the field of computer vision research. In this paper, we investigate the use of deep self-supervised learning methods to solve the problem of recogni…
It is difficult to train classifiers on paintings collections due to model bias from domain gaps and data bias from the uneven distribution of artistic styles. Previous techniques like data distillation, traditional data augmentation and style transfer improve classifier training using task specific training datasets or domain adaptation. We propose a system to handle data bias in small paintings datasets like the Kaokore dataset while simultaneously accounting for domain adaptation in fine-tuning a model trained on real world images. Our system consists of two stages which are style transfer …
University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；NetEase Fuxi AI Lab（网易伏羲人工智能实验室）；Beihang University（北京航空航天大学）
This paper proposes an image-to-painting translation method that generates vivid and realistic painting artworks with controllable styles. Different from previous image-to-image translation methods that formulate the translation as pixel-wise prediction, we deal with such an artistic creation process in a vectorized environment and produce a sequence of physically meaningful stroke parameters that can be further used for rendering. Since a typical vector render is not differentiable, we design a novel neural renderer which imitates the behavior of the vector renderer and then frame the stroke …
In this paper, we analyze a sample of clippings from paintings by the late artist Bob Ross. Previous work focused on the qualitative themes of his paintings (Hickey, 2014); here, we expand on that line of research by considering the colorspace and luminosity values as our data. Our results demonstrate the subtle aesthetics of the average Ross painting, the common variation shared by his paintings, and the structure of the relationships between each painting in our sample. We reveal, for the first time, renderings of the average paintings and introduce "eigenross" components to identify and eva…
CreativeSynth: Cross-Art-Attention for Artistic Image Synthesis with Multimodal Diffusion
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；MAIS, Institute of Automation, Chinese Academy of Sciences（自动化所MAIS部）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；University of Chinese Academy of Sciences（中国科学院大学）；ByteDance Inc.（字节跳动公司）；National Cheng Kung University（国立成功大学）
POSTA: A Go-to Framework for Customized Artistic Poster Generation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；The Hong Kong University of Science and Technology（香港科技大学）
