Image aesthetic quality assessment has got much attention in recent years, but not many works have been done on a specific genre of photos: Group photograph. In this work, we designed a set of high-level features based on the experience and principles of group photography: Opened-eye, Gaze, Smile, Occluded faces, Face Orientation, Facial blur, Character center. Then we combined them and 83 generic aesthetic features to build two aesthetic assessment models. We also constructed a large dataset of group photographs - GPD- annotated with the aesthetic score. The experimental result shows that our features perform well for categorizing professional photos and snapshots and predicting the distinction of multiple group photographs of diverse human states under the same scene.
Beihang University（北京航空航天大学）；Beijing Electronic Science and Technology Institute（北京电子科技学院）；Peng Cheng Laboratory（鹏城实验室）
Despite the growing number of unmanned aerial vehicles (UAVs) and aerial videos, there is a paucity of studies focusing on the aesthetics of aerial videos that can provide valuable information for improving the aesthetic quality of aerial photography. In this article, we present a method of deep multimodality learning for UAV video aesthetic quality assessment. More specifically, a multistream framework is designed to exploit aesthetic attributes from multiple modalities, including spatial appearance, drone camera motion, and scene structure. A novel specially designed motion stream network is…
Beijing Electronic Science and Technology Institute（北京电子科技学院）；CETC Big Data Research Institute Co.,Ltd.（中电科大数据研究院有限公司）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；SenseTime Research（商汤科技研究院）；Beihang University（北京航空航天大学）；Peng Cheng Laboratory（鹏城实验室）
Image aesthetic quality assessment has been a relatively hot topic during the last decade. Most recently, comments type assessment (aesthetic captions) has been proposed to describe the general aesthetic impression of an image using text. In this paper, we propose Aesthetic Attributes Assessment of Images, which means the aesthetic attributes captioning. This is a new formula of image aesthetic assessment, which predicts aesthetic attributes captions together with the aesthetic score of each attribute. We introduce a new dataset named \emph{DPC-Captions} which contains comments of up to 5 aest…
University of Science and Technology of China（中国科学技术大学）；Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；University of California, Davis（加州大学戴维斯分校）
Image aesthetic quality assessment is popular during the last decade. Besides numerical assessment, nature language assessment (aesthetic captioning) has been proposed to describe the generally aesthetic impression of an image. In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i.e., to assess the aesthetic attributes such as composition, lighting usage and color arrangement. It is a non-trivial task to label the comments of aesthetic attributes, which limit the scale of the corresponding datasets. We construct a novel dataset, named DPC-Cap…
Beijing Electronic Science and Technology Institute（北京电子科技学院）；University of California, Davis（加州大学戴维斯分校）；National Institute of Technology Patna（巴特那国家理工学院）
With the continuous development of social software and multimedia technology, images have become a kind of important carrier for spreading information and socializing. How to evaluate an image comprehensively has become the focus of recent researches. The traditional image aesthetic assessment methods often adopt single numerical overall assessment scores, which has certain subjectivity and can no longer meet the higher aesthetic requirements. In this paper, we construct an new image attribute dataset called aesthetic mixed dataset with attributes(AMD-A) and design external attribute features …
Image aesthetics assessment (IAA) is a challenging task due to its highly subjective nature. Most of the current studies rely on large-scale datasets (e.g., AVA and AADB) to learn a general model for all kinds of photography images. However, little light has been shed on measuring the aesthetic quality of artistic images, and the existing datasets only contain relatively few artworks. Such a defect is a great obstacle to the aesthetic assessment of artistic images. To fill the gap in the field of artistic image aesthetics assessment (AIAA), we first introduce a large-scale AIAA dataset: Boldbr…
Friedrich Schiller University（弗里德里希·席勒大学）；University of Jena School of Medicine（耶拿大学医学院）；Jena University Hospital（耶拿大学医院）
Computational aesthetics is an emerging field of research which has attracted different research groups in the last few years. In this field, one of the main approaches to evaluate the aesthetic quality of paintings and photographs is a feature-based approach. Among the different features proposed to reach this goal, color plays an import role. In this paper, we introduce a novel dataset that consists of paintings of Western provenance from 36 well-known painters from the 15th to the 20th century. As a first step and to assess this dataset, using a classifier, we investigate the correlation be…
The aesthetic quality of an image is defined as the measure or appreciation of the beauty of an image. Aesthetics is inherently a subjective property but there are certain factors that influence it such as, the semantic content of the image, the attributes describing the artistic aspect, the photographic setup used for the shot, etc. In this paper we propose a method for the automatic prediction of the aesthetics of an image that is based on the analysis of the semantic content, the artistic style and the composition of the image. The proposed network includes: a pre-trained network for semant…
Media Technology Center (MTC)（媒体技术中心（MTC））；ETH Zurich（苏黎世联邦理工学院）；DISCo；University of Milano-Bicocca（米兰比可卡大学）
Computational inference of aesthetics is an ill-defined task due to its subjective nature. Many datasets have been proposed to tackle the problem by providing pairs of images and aesthetic scores based on human ratings. However, humans are better at expressing their opinion, taste, and emotions by means of language rather than summarizing them in a single number. In fact, photo critiques provide much richer information as they reveal how and why users rate the aesthetics of visual stimuli. In this regard, we propose the Reddit Photo Critique Dataset (RPCD), which contains tuples of image and p…
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing University of Chemical Technology（北京化工大学）；Chinese Academy of Sciences（中国科学院）
Aesthetic quality prediction is a challenging task in the computer vision community because of the complex interplay with semantic contents and photographic technologies. Recent studies on the powerful deep learning based aesthetic quality assessment usually use a binary high-low label or a numerical score to represent the aesthetic quality. However the scalar representation cannot describe well the underlying varieties of the human perception of aesthetics. In this work, we propose to predict the aesthetic score distribution (i.e., a score distribution vector of the ordinal basic human rating…
School of Information Science and Engineering, Xiamen University（厦门大学信息科学与技术学院）；Institute of Information Science, Academia Sinica（中央研究院信息科学研究所）
Despite recent progress, computational visual aesthetic is still challenging. Image cropping, which refers to the removal of unwanted scene areas, is an important step to improve the aesthetic quality of an image. However, it is challenging to evaluate whether cropping leads to aesthetically pleasing results because the assessment is typically subjective. In this paper, we propose a novel cascaded cropping regression (CCR) method to perform image cropping by learning the knowledge from professional photographers. The proposed CCR method improves the convergence speed of the cascaded method, wh…
Image composition assessment is crucial in aesthetic assessment, which aims to assess the overall composition quality of a given image. However, to the best of our knowledge, there is neither dataset nor method specifically proposed for this task. In this paper, we contribute the first composition assessment dataset CADB with composition scores for each image provided by multiple professional raters. Besides, we propose a composition assessment network SAMP-Net with a novel Saliency-Augmented Multi-pattern Pooling (SAMP) module, which analyses visual layout from the perspectives of multiple co…
DEAR: Dataset for Evaluating the Aesthetics of Rendering
Color Reproduction and Synthesis Institute（颜色再现与合成研究所）；Moscow Independent Research Institute of Artificial Intelligence（莫斯科独立人工智能研究 institute）；Computational color photography Laboratory（计算色彩摄影实验室）；University of Milano-Bicocca（米兰-比科卡大学）
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；University of California, Davis（加州大学戴维斯分校）
In the tasks of image aesthetic quality evaluation, it is difficult to reach both the high score area and low score area due to the normal distribution of aesthetic datasets. To reduce the error in labeling and solve the problem of normal data distribution, we propose a new aesthetic mixed dataset with classification and regression called AMD-CR, and we train a meta reweighting network to reweight the loss of training data differently. In addition, we provide a training strategy acccording to different stages, based on pseudo labels of the binary classification task, and then we use it for aes…
Evaluating aesthetic value of digital photographs is a challenging task, mainly due to numerous factors that need to be taken into account and subjective manner of this process. In this paper, we propose to approach this problem using deep convolutional neural networks. Using a dataset of over 1.7 million photos collected from Flickr, we train and evaluate a deep learning model whose goal is to classify input images by analysing their aesthetic value. The result of this work is a publicly available Web-based application that can be used in several real-life applications, e.g. to improve the wo…
Beijing Electronic Science and Technology Institute（北京电子科技学院）；University of Science and Technology of China（中国科学技术大学）；University of California, Davis（加州大学戴维斯分校）
Aesthetic assessment of images can be categorized into two main forms: numerical assessment and language assessment. Aesthetics caption of photographs is the only task of aesthetic language assessment that has been addressed. In this paper, we propose a new task of aesthetic language assessment: aesthetic visual question and answering (AVQA) of images. If we give a question of images aesthetics, model can predict the answer. We use images from \textit{www.flickr.com}. The objective QA pairs are generated by the proposed aesthetic attributes analysis algorithms. Moreover, we introduce subjectiv…
When assessing whether an image is of high or low quality, it is indispensable to take personal preference into account. Existing aesthetic models lay emphasis on hand-crafted features or deep features commonly shared by high quality images, but with limited or no consideration for personal preference and user interaction. To that end, we propose a novel and user-friendly aesthetic ranking framework via powerful deep neural network and a small amount of user interaction, which can automatically estimate and rank the aesthetic characteristics of images in accordance with users' preference. Our …
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；School of Information and Electronic Engineering, East China Normal University（华东师范大学信息与电子工程学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）
University of California, Davis（加利福尼亚大学戴维斯分校）；National Taiwan University（台湾大学）；National Tsing Hua University（清华大学（台湾））
Photo composition is an important factor affecting the aesthetics in photography. However, it is a highly challenging task to model the aesthetic properties of good compositions due to the lack of globally applicable rules to the wide variety of photographic styles. Inspired by the thinking process of photo taking, we formulate the photo composition problem as a view finding process which successively examines pairs of views and determines their aesthetic preferences. We further exploit the rich professional photographs on the web to mine unlimited high-quality ranking samples and demonstrate …
AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
University of California, Berkeley（加州大学伯克利分校）；Adobe（Adobe公司）
The style of an image plays a significant role in how it is viewed, but style has received little attention in computer vision research. We describe an approach to predicting style of images, and perform a thorough evaluation of different image features for these tasks. We find that features learned in a multi-layer network generally perform best -- even when trained with object class (not style) labels. Our large-scale learning methods results in the best published performance on an existing dataset of aesthetic ratings and photographic style annotations. We present two novel datasets: 80K Fl…
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）
Aesthetic quality assessment (AQA) is a challenging task due to complex aesthetic factors. Currently, it is common to conduct AQA using deep neural networks that require fixed-size inputs. Existing methods mainly transform images by resizing, cropping, and padding or employ adaptive pooling to alternately capture the aesthetic features from fixed-size inputs. However, these transformations potentially damage aesthetic features. To address this issue, we propose a simple but effective method to accomplish full-resolution image AQA by combining image padding with region of image (RoM) pooling. P…
APDDv2: Aesthetics of Paintings and Drawings Dataset with Artist Labeled Scores and Comments
Beijing Electronic Science and Technology Institute（北京电子科技学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Central Academy of Fine Arts（中央美术学院）；University of Science and Technology of China（中国科学技术大学）；Beijing University of Technology（北京工业大学）
Trinity College Dublin（都柏林圣三一学院）；Accenture Labs（埃森哲实验室）
Aspect ratio and spatial layout are two of the principal factors determining the aesthetic value of a photograph. But, incorporating these into the traditional convolution-based frameworks for the task of image aesthetics assessment is problematic. The aspect ratio of the photographs gets distorted while they are resized/cropped to a fixed dimension to facilitate training batch sampling. On the other hand, the convolutional filters process information locally and are limited in their ability to model the global spatial layout of a photograph. In this work, we present a two-stage framework base…
Machine-learning excels in many areas with well-defined goals. However, a clear goal is usually not available in art forms, such as photography. The success of a photograph is measured by its aesthetic value, a very subjective concept. This adds to the challenge for a machine learning approach. We introduce Creatism, a deep-learning system for artistic content creation. In our system, we break down aesthetics into multiple aspects, each can be learned individually from a shared dataset of professional examples. Each aspect corresponds to an image operation that can be optimized efficiently. A …
International Institute of Information Technology（国际信息技术学院）；University of Hyderabad（海得拉巴大学）
Automatic photo aesthetic assessment is a challenging artificial intelligence task. Existing computational approaches have focused on modeling a single aesthetic score or a class (good or bad), however these do not provide any details on why the photograph is good or bad, or which attributes contribute to the quality of the photograph. To obtain both accuracy and human interpretation of the score, we advocate learning the aesthetic attributes along with the prediction of the overall score. For this purpose, we propose a novel multitask deep convolution neural network, which jointly learns eigh…
V-SENSE School of Computer Science Statistics Trinity College Dublin（V-SENSE计算机科学与统计学院都柏林大学三一学院）
Aesthetic image captioning (AIC) refers to the multi-modal task of generating critical textual feedbacks for photographs. While in natural image captioning (NIC), deep models are trained in an end-to-end manner using large curated datasets such as MS-COCO, no such large-scale, clean dataset exists for AIC. Towards this goal, we propose an automatic cleaning strategy to create a benchmarking AIC dataset, by exploiting the images and noisy comments easily available from photography websites. We propose a probabilistic caption-filtering method for cleaning the noisy web-data, and compile a large-…
Many photography websites such as Flickr, 500px, Unsplash, and Adobe Behance are used by amateur and professional photography enthusiasts. Unlike content-based image search, such users of photography websites are not just looking for photos with certain content, but more generally for photos with a certain photographic "aesthetic". In this context, we explore personalized photo recommendation and propose two aesthetic feature extraction methods based on (i) color space and (ii) deep style transfer embeddings. Using a dataset from 500px, we evaluate how these features can be best leveraged by c…
The capacity of automatically modeling photographic composition is valuable for many real-world machine vision applications such as digital photography, image retrieval, image understanding, and image aesthetics assessment. The triangle technique is among those indispensable composition methods on which professional photographers often rely. This paper proposes a system that can identify prominent triangle arrangements in two major categories of photographs: natural or urban scenes, and portraits. For the natural or urban scene pictures, the focus is on the effect of linear perspective. For po…
Aesthetic assessment is subjective, and the distribution of the aesthetic levels is imbalanced. In order to realize the auto-assessment of photo aesthetics, we focus on retraining the CNN-based aesthetic assessment model by dropping out the unavailable samples in the middle levels from the training data set repetitively to overcome the effect of imbalanced aesthetic data on classification. Further, the method of extracting aesthetics highlight region of the photo image by using the two repetitively trained models is presented. Therefore, the correlation of the extracted region with the aesthet…
Recent research has widely explored the problem of aesthetics assessment of images with generic content. However, few approaches have been specifically designed to predict the aesthetic quality of images containing human faces, which make up a massive portion of photos in the web. This paper introduces a method for aesthetic quality assessment of images with faces. We exploit three different Convolutional Neural Networks to encode information regarding perceptual quality, global image aesthetics, and facial attributes; then, a model is trained to combine these features to explicitly predict th…
Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?
Bake AI；University of Washington（华盛顿大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Stanford University（斯坦福大学）；University of Notre Dame（诺丁汉大学）；Carnegie Mellon University（卡内基梅隆大学）；MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）；Western Washington University（西雅图华盛顿大学）；King Abdulaziz City for Science and Technology（国王阿卜杜勒阿齐兹科技城）
Researchers try to model the aesthetic quality of photographs into low and high- level features, drawing inspiration from art theory, psychology and marketing. We attempt to describe every feature extraction measure employed in the above process. The contribution of this literature review is the taxonomy of each feature by its implementation complexity, considering real-world applications and integration in mobile apps and digital cameras. Also, we discuss the machine learning results along with some unexplored research areas as future work.
Beijing Lab of Intelligent Information Technology（北京市智能信息技术实验室）；School of Computer Science, Beijing Institute of Technology（北京理工大学计算机学院）
We model the photo cropping problem as a cascade of attention box regression and aesthetic quality classification, based on deep learning. A neural network is designed that has two branches for predicting attention bounding box and analyzing aesthetics, respectively. The predicted attention box is treated as an initial crop window where a set of cropping candidates are generated around it, without missing important information. Then, aesthetics assessment is employed to select the final crop as the one with the best aesthetic quality. With our network, cropping candidates share features within…
Predicting Scores of Various Aesthetic Attribute Sets by Learning from Overall Score Labels
University of Science and Technology of China（中国科学技术大学）；Beijing Electronic Science and Technology Institute（北京电子科技学院）；University of Edinburgh（爱丁堡大学）；Sensetime Research（商汤科技研究院）
Nanyang Technological University（南洋理工大学）；DSO National Laboratories（DSO国家实验室）；Alibaba Group（阿里巴巴集团）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-南洋理工大学新加坡联合研究院）
This survey aims at reviewing recent computer vision techniques used in the assessment of image aesthetic quality. Image aesthetic assessment aims at computationally distinguishing high-quality photos from low-quality ones based on photographic rules, typically in the form of binary classification or quality scoring. A variety of approaches has been proposed in the literature trying to solve this challenging problem. In this survey, we present a systematic listing of the reviewed approaches based on visual feature types (hand-crafted features and deep features) and evaluation criteria (dataset…
University of Engineering and Technology, Peshawar（白沙瓦工程技术大学）；COMSATS University Islamabad, Wah Campus, Pakistan（伊斯兰堡COMSATS大学瓦赫校区）；Saudi Electronic University, Saudi Arabia（沙特电子大学）；Imaging and Computer vision group, Data61-CSIRO, Australia（澳大利亚CSIRO数据61中心成像与计算机视觉组）；Norwegian University of Life Sciences, Norway（挪威生命科学大学）
Automatic image aesthetics assessment is a computer vision problem dealing with categorizing images into different aesthetic levels. The categorization is usually done by analyzing an input image and computing some measure of the degree to which the image adheres to the fundamental principles of photography such as balance, rhythm, harmony, contrast, unity, look, feel, tone and texture. Due to its diverse applications in many areas, automatic image aesthetic assessment has gained significant research attention in recent years. This article presents a review of the contemporary automatic image …
Deep convolutional neural networks have recently achieved great success on image aesthetics assessment task. In this paper, we propose an efficient method which takes the global, local and scene-aware information of images into consideration and exploits the composite features extracted from corresponding pretrained deep learning models to classify the derived features with support vector machine. Contrary to popular methods that require fine-tuning or training a new model from scratch, our training-free method directly takes the deep features generated by off-the-shelf models for image classi…
SUNY at Buffalo（布法罗纽约州立大学）；Tianjin University（天津大学）
Deep convolutional neural networks (CNN) have recently been shown to generate promising results for aesthetics assessment. However, the performance of these deep CNN methods is often compromised by the constraint that the neural network only takes the fixed-size input. To accommodate this requirement, input images need to be transformed via cropping, warping, or padding, which often alter image composition, reduce image resolution, or cause image distortion. Thus the aesthetics of the original images is impaired because of potential loss of fine grained details and holistic image layout. Howev…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Youtu Lab, Tencent（腾讯优图实验室）；Snap Inc.（Snap公司）；North China Electric Power University（华北电力大学）；Xiamen University（厦门大学）；Kuaishou Technology（快手科技）
Visual aesthetic assessment has been an active research field for decades. Although latest methods have achieved promising performance on benchmark datasets, they typically rely on a large number of manual annotations including both aesthetic labels and related image attributes. In this paper, we revisit the problem of image aesthetic assessment from the self-supervised feature learning perspective. Our motivation is that a suitable feature representation for image aesthetic assessment should be able to distinguish different expert-designed image manipulations, which have close relationships w…
