No-Reference Image Quality Assessment (NR-IQA) focuses on designing methods to measure image quality in alignment with human perception when a high-quality reference image is unavailable. Most state-of-the-art NR-IQA approaches are opinion-aware, i.e. they require human annotations for training. This dependency limits their scalability and broad applicability. To overcome this limitation, we propose QualiCLIP (Quality-aware CLIP), a CLIP-based self-supervised opinion-unaware approach that does not require human opinions. In particular, we introduce a quality-aware image-text alignment strategy to make CLIP generate quality-aware image representations. Starting from pristine images, we synthetically degrade them with increasing levels of intensity. Then, we train CLIP to rank these degraded images based on their similarity to quality-related antonym text prompts. At the same time, we force CLIP to generate consistent representations for images with similar content and the same level of degradation. Our experiments show that the proposed method improves over existing opinion-unaware approaches across multiple datasets with diverse distortion types. Moreover, despite not requiring human annotations, QualiCLIP achieves excellent performance against supervised opinion-aware methods in cross-dataset experiments, thus demonstrating remarkable generalization capabilities. The code and the model are publicly available at https://github.com/miccunifi/QualiCLIP.
University of Alberta（阿尔伯塔大学）；CNRS（法国国家科学研究中心）；CentraleSupelec（中央理工-高等电力学院）；Université Paris-Sud（巴黎第十一大学）
Image Quality Assessment algorithms predict a quality score for a pristine or distorted input image, such that it correlates with human opinion. Traditional methods required a non-distorted "reference" version of the input image to compare with, in order to predict this score. However, recent "No-reference" methods circumvent this requirement by modelling the distribution of clean image features, thereby making them more suitable for practical use. However, majority of such methods either use hand-crafted features or require training on human opinion scores (supervised learning), which are dif…
Viewport-Unaware Blind Omnidirectional Image Quality Assessment: A Flexible and Effective Paradigm
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算与人工智能学院）；Harvard Medical School, Harvard University（哈佛大学医学院）
Viewport-Unaware Blind Omnidirectional Image Quality Assessment: A Unified and Generalized Approach
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Jiangxi AI Quality Testing and Inspection Center（江西省人工智能质量检测与检验中心）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）
Recent learning-based video quality assessment (VQA) algorithms are expensive to implement due to the cost of data collection of human quality opinions, and are less robust across various scenarios due to the biases of these opinions. This motivates our exploration on opinion-unaware (a.k.a zero-shot) VQA approaches. Existing approaches only considers low-level naturalness in spatial or temporal domain, without considering impacts from high-level semantics. In this work, we introduce an explicit semantic affinity index for opinion-unaware VQA using text-prompts in the contrastive language-imag…
GMC-IQA: Exploiting Global-correlation and Mean-opinion Consistency for No-reference Image Quality Assessment
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shenzhen Heytap Technology Corp., Ltd.（深圳嘿哟科技有限公司）；Alibaba Group（阿里巴巴集团）；Nanchang Hangkong University（南昌航空大学）
Deep-learning based techniques have contributed to the remarkable progress in the field of automatic image quality assessment (IQA). Existing IQA methods are designed to measure the quality of an image in terms of Mean Opinion Score (MOS) at the image-level (i.e. the whole image) or at the patch-level (dividing the image into multiple units and measuring quality of each patch). Some applications may require assessing the quality at the pixel-level (i.e. MOS value for each pixel), however, this is not possible in case of existing techniques as the spatial information is lost owing to their netw…
Objective image quality assessment deals with the prediction of digital images' perceptual quality. No-reference image quality assessment predicts the quality of a given input image without any knowledge or information about its pristine (distortion free) counterpart. Machine learning algorithms are heavily used in no-reference image quality assessment because it is very complicated to model the human visual system's quality perception. Moreover, no-reference image quality assessment algorithms are evaluated on publicly available benchmark databases. These databases contain images with their c…
Institute of Image Communication and Information Processing, Shanghai Jiao Tong University（上海交通大学图像通信与信息处理研究所）；BOCOM Smart Network Technologies Inc.（交银智能网络技术有限公司）
To provide the possibility of developing objective image quality assessment (IQA) algorithms for THz security images, we constructed the THz security image database (THSID) including a total of 181 THz security images with the resolution of 127*380. The main distortion types in THz security images were first analyzed for the design of subjective evaluation criteria to acquire the mean opinion scores. Subsequently, the existing no-reference IQA algorithms, which were 5 opinion-aware approaches viz., NFERM, GMLF, DIIVINE, BRISQUE and BLIINDS2, and 8 opinion-unaware approaches viz., QAC, SISBLIM,…
Universidade Federal do Espírito Santo（圣埃斯皮里图联邦大学）
This paper uses robust statistics and curvelet transform to learn a general-purpose no-reference (NR) image quality assessment (IQA) model. The new approach, here called M1, competes with the Curvelet Quality Assessment proposed in 2014 (Curvelet2014). The central idea is to use descriptors based on robust statistics to extract features and predict the human opinion about degraded images. To show the consistency of the method the model is tested with 3 different datasets, LIVE IQA, TID2013 and CSIQ. To test evaluation, it is used the Wilcoxon test to verify the statistical significance of resu…
University of Waterloo（滑铁卢大学）；UBTech Sydney Artificial Intelligence Institute（优必选悉尼人工智能研究院）；The University of Sydney（悉尼大学）
Objective assessment of image quality is fundamentally important in many image processing tasks. In this work, we focus on learning blind image quality assessment (BIQA) models which predict the quality of a digital image with no access to its original pristine-quality counterpart as reference. One of the biggest challenges in learning BIQA models is the conflict between the gigantic image space (which is in the dimension of the number of image pixels) and the extremely limited reliable ground truth data for training. Such data are typically collected via subjective testing, which is cumbersom…
Exploring Rich Subjective Quality Information for Image Quality Assessment in the Wild
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；Peng Cheng Laboratory (PCL)（鹏城实验室）；The Hong Kong Polytechnic University（香港理工大学）
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；ADAPT SFI Research Centre（ADAPT SFI研究中心）；University College Dublin（都柏林大学）；Nanyang Technological University Singapore（新加坡南洋理工大学）；National University of Singapore（新加坡国立大学）
Image quality assessment is critical to control and maintain the perceived quality of visual content. Both subjective and objective evaluations can be utilised, however, subjective image quality assessment is currently considered the most reliable approach. Databases containing distorted images and mean opinion scores are needed in the field of atmospheric research with a view to improve the current state-of-the-art methodologies. In this paper, we focus on using ground-based sky camera images to understand the atmospheric events. We present a new image quality assessment dataset containing or…
Ningbo University（宁波大学）；Ningbo Institute of Industrial Technology, Chinese Academy of Sciences（中国科学院宁波工业技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）
With the development of image recovery models,especially those based on adversarial and perceptual losses,the detailed texture portions of images are being recovered more naturally.However,these restored images are similar but not identical in detail texture to their reference images.With traditional image quality assessment methods,results with better subjective perceived quality often score lower in objective scoring.Assessment methods suffer from subjective and objective inconsistencies.This paper proposes a regional differential information entropy (RDIE) method for image quality assessmen…
University of Würzburg（维尔茨堡大学）；Institute of Computer Science（计算机科学研究所）
Generative models for image restoration, enhancement, and generation have significantly improved the quality of the generated images. Surprisingly, these models produce more pleasant images to the human eye than other methods, yet, they may get a lower perceptual quality score using traditional perceptual quality metrics such as PSNR or SSIM. Therefore, it is necessary to develop a quantitative metric to reflect the performance of new algorithms, which should be well-aligned with the person's mean opinion score (MOS). Learning-based approaches for perceptual image quality assessment (IQA) usua…
Xidian University（西安电子科技大学）；University of Southern California（南加州大学）
A new stereoscopic image quality assessment database rendered using the 2D-image-plus-depth source, called MCL-3D, is described and the performance benchmarking of several known 2D and 3D image quality metrics using the MCL-3D database is presented in this work. Nine image-plus-depth sources are first selected, and a depth image-based rendering (DIBR) technique is used to render stereoscopic image pairs. Distortions applied to either the texture image or the depth image before stereoscopic image rendering include: Gaussian blur, additive white noise, down-sampling blur, JPEG and JPEG-2000 (JP2…
Most publicly available image quality databases have been created under highly controlled conditions by introducing graded simulated distortions onto high-quality photographs. However, images captured using typical real-world mobile camera devices are usually afflicted by complex mixtures of multiple distortions, which are not necessarily well-modeled by the synthetic distortions found in existing databases. The originators of existing legacy databases usually conducted human psychometric studies to obtain statistically meaningful sets of human opinion scores on images in a stringently control…
University of Engineering and Technology Lahore（拉合尔工程技术大学）
The assessment of the perceptual quality of digital images is becoming increasingly important as a result of the widespread use of digital multimedia devices. Smartphones and high-speed internet are just two examples of technologies that have multiplied the amount of multimedia content available. Thus, obtaining a representative dataset, which is required for objective quality assessment training, is a significant challenge. The Blind Image Quality Assessment Database, BIQ2021, is presented in this article. By selecting images with naturally occurring distortions and reliable labeling, the dat…
Qingdao University（青岛大学）；Ocean University of China（中国海洋大学）
Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing. However, the development of underwater image quality assessment (UIQA) is limited for the lack of comprehensive human subjective user study with publicly available dataset and reliable objective UIQA metric. To address this issue, we establish a large-scale underwater image dataset, dubbed UID2021, for evaluating no-reference UIQA metrics. The constructed dataset contains 60 multiply degraded underwater images collected from various sour…
Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）
We present a no reference (NR) quality assessment algorithm for assessing the perceptual quality of natural stereoscopic 3D (S3D) videos. This work is inspired by our finding that the joint statistics of the subband coefficients of motion (optical flow or motion vector magnitude) and depth (disparity map) of natural S3D videos possess a unique signature. Specifically, we empirically show that the joint statistics of the motion and depth subband coefficients of S3D video frames can be modeled accurately using a Bivariate Generalized Gaussian Distribution (BGGD). We then demonstrate that the par…
The quality assessment (QA) of restored low light images is an important tool for benchmarking and improving low light restoration (LLR) algorithms. While several LLR algorithms exist, the subjective perception of the restored images has been much less studied. Challenges in capturing aligned low light and well-lit image pairs and collecting a large number of human opinion scores of quality for training, warrant the design of unsupervised (or opinion unaware) no-reference (NR) QA methods. This work studies the subjective perception of low light restored images and their unsupervised NR QA. Our…
MobileIQA: Exploiting Mobile-level Diverse Opinion Network For No-Reference Image Quality Assessment Using Knowledge Distillation
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Union University（北京联合大学）；China University of Petroleum（中国石油大学）；Hebei University（河北大学）；PeopleAl Inc.（北京人智科技有限公司）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Transsion Information Technology Limited（上海传音信息技术有限公司）
Automatically learned quality assessment for images has recently become a hot topic due to its usefulness in a wide variety of applications such as evaluating image capture pipelines, storage techniques and sharing media. Despite the subjective nature of this problem, most existing methods only predict the mean opinion score provided by datasets such as AVA [1] and TID2013 [2]. Our approach differs from others in that we predict the distribution of human opinion scores using a convolutional neural network. Our architecture also has the advantage of being significantly simpler than other method…
The rapid increase in user-generated-content (UGC) videos calls for the development of effective video quality assessment (VQA) algorithms. However, the objective of the UGC-VQA problem is still ambiguous and can be viewed from two perspectives: the technical perspective, measuring the perception of distortions; and the aesthetic perspective, which relates to preference and recommendation on contents. To understand how these two perspectives affect overall subjective opinions in UGC-VQA, we conduct a large-scale subjective study to collect human quality opinions on overall quality of videos as…
Subjective and Objective Quality Assessment Methods of Stereoscopic Videos with Visibility Affecting Distortions
Indian Institute of Information Technology, Design and Manufacturing Kancheepuram（印度信息技术、设计与制造坎奇普兰学院）；Indian Institute of Technology Indore（印度印多尔理工学院）；Indian Institute of Technology Hyderabad（印度海得拉巴理工学院）
Institute of Technology, Sligo（斯莱戈理工学院）；Dublin City University（都柏林城市大学）
Automatic colourisation of grey-scale images is the process of creating a full-colour image from the grey-scale prior. It is an ill-posed problem, as there are many plausible colourisations for a given grey-scale prior. The current SOTA in auto-colourisation involves image-to-image type Deep Convolutional Neural Networks with Generative Adversarial Networks showing the greatest promise. The end goal of colourisation is to produce full colour images that appear plausible to the human viewer, but human assessment is costly and time consuming. This work assesses how well commonly used objective m…
Univ Lyon, CNRS, LIRIS（里昂大学，法国国家科学研究中心，LIRIS）；Multimedia Research Center, University of Alberta（阿尔伯塔大学多媒体研究中心）
Objective visual quality assessment of 3D models is a fundamental issue in computer graphics. Quality assessment metrics may allow a wide range of processes to be guided and evaluated, such as level of detail creation, compression, filtering, and so on. Most computer graphics assets are composed of geometric surfaces on which several texture images can be mapped to 11 make the rendering more realistic. While some quality assessment metrics exist for geometric surfaces, almost no research has been conducted on the evaluation of texture-mapped 3D models. In this context, we present a new subject…
Indian Institute of Technology Jodhpur（焦特布尔印度理工学院）；International Institute of Information Technology（国际信息技术研究所）
Automated vision-based score estimation models can be used as an alternate opinion to avoid judgment bias. In the past works the score estimation models were learned by regressing the video representations to the ground truth score provided by the judges. However such regression-based solutions lack interpretability in terms of giving reasons for the awarded score. One solution to make the scores more explicable is to compare the given action video with a reference video. This would capture the temporal variations w.r.t. the reference video and map those variations to the final score. In this …
Guangzhou University（广州大学）；Guangdong University of Education（广东第二师范学院）；Xidian University（西安电子科技大学）；Harbin Institute of Technology（哈尔滨工业大学）
The attention mechanism is blooming in computer vision nowadays. However, its application to video quality assessment (VQA) has not been reported. Evaluating the quality of in-the-wild videos is challenging due to the unknown of pristine reference and shooting distortion. This paper presents a novel \underline{s}pace-\underline{t}ime \underline{a}ttention network fo\underline{r} the \underline{VQA} problem, named StarVQA. StarVQA builds a Transformer by alternately concatenating the divided space-time attention. To adapt the Transformer architecture for training, StarVQA designs a vectorized r…
City University of Hong Kong（香港城市大学）；Tongji University（同济大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）
Recent years have witnessed the dramatically increased interest in face generation with generative adversarial networks (GANs). A number of successful GAN algorithms have been developed to produce vivid face images towards different application scenarios. However, little work has been dedicated to automatic quality assessment of such GAN-generated face images (GFIs), even less have been devoted to generalized and robust quality assessment of GFIs generated with unseen GAN model. Herein, we make the first attempt to study the subjective and objective quality towards generalized quality assessme…
Nagoya University（名古屋大学）；National University of Singapore（新加坡国立大学）
AI-driven Action Quality Assessment (AQA) of sports videos can mimic Olympic judges to help score performances as a second opinion or for training. However, these AI methods are uninterpretable and do not justify their scores, which is important for algorithmic accountability. Indeed, to account for their decisions, instead of scoring subjectively, sports judges use a consistent set of criteria - rubric - on multiple actions in each performance sequence. Therefore, we propose IRIS to perform Interpretable Rubric-Informed Segmentation on action sequences for AQA. We investigated IRIS for scorin…
Millions of cameras at edge are being deployed to power a variety of different deep learning applications. However, the frames captured by these cameras are not always pristine - they can be distorted due to lighting issues, sensor noise, compression etc. Such distortions not only deteriorate visual quality, they impact the accuracy of deep learning applications that process such video streams. In this work, we introduce AQuA, to protect application accuracy against such distorted frames by scoring the level of distortion in the frames. It takes into account the analytical quality of frames, n…
Modeling Beyond MOS: Quality Assessment Models Must Integrate Context, Reasoning, and Multimodality
R&D Lab F-Initiatives（F-Initiatives研发实验室）；Université d’Orleans（奥尔良大学）；Université Sorbonne Paris Nord（巴黎-索邦大学）；University of Dubai（迪拜大学）；IULM University（IULM大学）
ESVQA: Perceptual Quality Assessment of Egocentric Spatial Videos
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；Institut Universitaire de France (IUF), University of Nantes（法国南特大学机构大学（IUF））
VMAF is a machine learning based video quality assessment method, originally designed for streaming applications, which combines multiple quality metrics and video features through SVM regression. It offers higher correlation with subjective opinions compared to many conventional quality assessment methods. In this paper we propose enhancements to VMAF through the integration of new video features and alternative quality metrics (selected from a diverse pool) alongside multiple model combination. The proposed combination approach enables training on multiple databases with varying content and …
