Increasing popularity of high-dynamic-range (HDR) image and video content brings the need for metrics that could predict the severity of image impairments as seen on displays of different brightness levels and dynamic range. Such metrics should be trained and validated on a sufficiently large subjective image quality dataset to ensure robust performance. As the existing HDR quality datasets are limited in size, we created a Unified Photometric Image Quality dataset (UPIQ) with over 4,000 images by realigning and merging existing HDR and standard-dynamic-range (SDR) datasets. The realigned quality scores share the same unified quality scale across all datasets. Such realignment was achieved by collecting additional cross-dataset quality comparisons and re-scaling data with a psychometric scaling method. Images in the proposed dataset are represented in absolute photometric and colorimetric units, corresponding to light emitted from a display. We use the new dataset to retrain existing HDR metrics and show that the dataset is sufficiently large for training deep architectures. We show the utility of the dataset on brightness aware image compression.
Can learning to measure the quality of an action help in measuring the quality of other actions? If so, can consolidated samples from multiple actions help improve the performance of current approaches? In this paper, we carry out experiments to see if knowledge transfer is possible in the action quality assessment (AQA) setting. Experiments are carried out on our newly released AQA dataset (http://rtis.oit.unlv.edu/datasets.html) consisting of 1106 action samples from seven actions with quality scores as measured by expert human judges. Our experimental results show that there is utility in l…
ISP RAS（俄罗斯科学院信息传输问题研究所）；Research Center for Trusted Artificial Intelligence（可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）；Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）
Video-quality measurement is a critical task in video processing. Nowadays, many implementations of new encoding standards - such as AV1, VVC, and LCEVC - use deep-learning-based decoding algorithms with perceptual metrics that serve as optimization objectives. But investigations of the performance of modern video- and image-quality metrics commonly employ videos compressed using older standards, such as AVC. In this paper, we present a new benchmark for video-quality metrics that evaluates video compression. It is based on a new dataset consisting of about 2,500 streams encoded using differen…
Ti-Patch: Tiled Physical Adversarial Patch for no-reference video quality metrics
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；Ivannikov Institute for System Programming of the RAS（俄罗斯科学院伊万尼科夫系统编程研究所）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能学院）
Research on video frame interpolation has made significant progress in recent years. However, existing methods mostly use off-the-shelf metrics to measure the quality of interpolation results with the exception of a few methods that employ user studies, which is time-consuming. As video frame interpolation results often exhibit unique artifacts, existing quality metrics sometimes are not consistent with human perception when measuring the interpolation results. Some recent deep learning-based perceptual quality metrics are shown more consistent with human judgments, but their performance on vi…
Super-resolution results are usually measured by full-reference image quality metrics or human rating scores. However, these evaluation methods are general image quality measurement, and do not account for the nature of the super-resolution problem. In this work, we analyze the evaluation problem based on the one-to-many mapping nature of super-resolution, and propose a novel distribution-based metric for super-resolution. Starting from the distribution distance, we derive the proposed metric to make it accessible and easy to compute. Through a human subject study on super-resolution, we show …
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；King’s College London（伦敦国王学院）
Image Quality Assessment (IQA) metrics are widely used to quantitatively estimate the extent of image degradation following some forming, restoring, transforming, or enhancing algorithms. We present PyTorch Image Quality (PIQ), a usability-centric library that contains the most popular modern IQA algorithms, guaranteed to be correctly implemented according to their original propositions and thoroughly verified. In this paper, we detail the principles behind the foundation of the library, describe the evaluation strategy that makes it reliable, provide the benchmarks that showcase the performan…
Budapest University of Technology and Economics（布达佩斯技术与经济大学）
In this study, our goal is to give a comprehensive evaluation of 32 state-of-the-art FR-IQA metrics using the recently published MDID. This database contains distorted images derived from a set of reference, pristine images using random types and levels of distortions. Specifically, Gaussian noise, Gaussian blur, contrast change, JPEG noise, and JPEG2000 noise were considered.
University of Westminster（威斯敏斯特大学）；NVIDIA Corporation（英伟达公司）
Spatial image quality metrics designed for camera systems generally employ the Modulation Transfer Function (MTF), the Noise Power Spectrum (NPS), and a visual contrast detection model. Prior art indicates that scene-dependent characteristics of non-linear, content-aware image processing are unaccounted for by MTFs and NPSs measured using traditional methods. We present two novel metrics: the log Noise Equivalent Quanta (log NEQ) and Visual log NEQ. They both employ scene-and-process-dependent MTF (SPD-MTF) and NPS (SPD-NPS) measures, which account for signal-transfer and noise scene-dependenc…
IOI: Invisible One-Iteration Adversarial Attack on No-Reference Image- and Video-Quality Metrics
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息学研究所可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）
Thapar Institute of Engineering and Technology（塔帕尔工程技术学院）；Oxford Brookes University（牛津布鲁克斯大学）
With the inclusion of camera in daily life, an automatic no reference image quality evaluation index is required for automatic classification of images. The present manuscripts proposes a new No Reference Regional Mutual Information based technique for evaluating the quality of an image. We use regional mutual information on subsets of the complete image. Proposed technique is tested on four benchmark natural image databases, and one benchmark synthetic database. A comparative analysis with classical and state-of-art methods indicate superiority of the present technique for high quality images…
Fast Adversarial CNN-based Perturbation Attack on No-Reference Image- and Video-Quality Metrics
Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）
Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks
MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）
Although several 2D quality metrics have been proposed for images and videos, in the case of 3D efforts are only at the initial stages. In this paper, we propose a new full-reference quality metric for 3D content. Our method is modeled around the HVS, fusing the information of both left and right channels, considering color components, the cyclopean views of the two videos and disparity. Performance evaluations showed that our 3D quality metric successfully monitors the degradation of quality caused by several representative types of distortion and it has 86% correlation with the results of su…
As the evolution of multiview display technology is bringing glasses-free 3DTV closer to reality, MPEG and VCEG are preparing an extension to HEVC to encode multiview video content. View synthesis in the current version of the 3D video codec is performed using PSNR as a quality metric measure. In this paper, we propose a full- reference Human-Visual-System based 3D video quality metric to be used in multiview encoding as an alternative to PSNR. Performance of our metric is tested in a 2-view case scenario. The quality of the compressed stereo pair, formed from a decoded view and a synthesized …
While there exists a wide variety of Low Dynamic Range (LDR) quality metrics, only a limited number of metrics are designed specifically for the High Dynamic Range (HDR) content. With the introduction of HDR video compression standardization effort by international standardization bodies, the need for an efficient video quality metric for HDR applications has become more pronounced. The objective of this study is to compare the performance of the existing full-reference LDR and HDR video quality metrics on HDR content and identify the most effective one for HDR applications. To this end, a new…
This contribution proposes a full-reference Human-Visual-System based 3D video quality metric. In this report, the presented metric is used to evaluate the quality of compressed stereo pair formed from a decoded view and a synthesized view. The performance of the proposed metric is verified through a series of subjective tests and compared with that of PSNR, SSIM, MS-SSIM, VIFp, and VQM metrics. The experimental results show that HV3D has the highest correlation with Mean Opinion Scores (MOS) compared to other tested metrics.
Point cloud coding solutions have been recently standardized to address the needs of multiple application scenarios. The design and assessment of point cloud coding methods require reliable objective quality metrics to evaluate the level of degradation introduced by compression or any other type of processing. Several point cloud objective quality metrics has been recently proposed to reliable estimate human perceived quality, including the so-called projection-based metrics. In this context, this paper proposes a joint geometry and color projection-based point cloud objective quality metric w…
Stereoscopic video technologies have been introduced to the consumer market in the past few years. A key factor in designing a 3D system is to understand how different visual cues and distortions affect the perceptual quality of stereoscopic video. The ultimate way to assess 3D video quality is through subjective tests. However, subjective evaluation is time consuming, expensive, and in some cases not possible. The other solution is developing objective quality metrics, which attempt to model the Human Visual System (HVS) in order to assess perceptual quality. Although several 2D quality metri…
In this paper, we propose a new full-reference quality metric for mobile 3D content. Our method is modeled around the Human Visual System, fusing the information of both left and right channels, considering color components, the cyclopean views of the two videos and disparity. Our method is assessing the quality of 3D videos displayed on a mobile 3DTV, taking into account the effect of resolution, distance from the viewers eyes, and dimensions of the mobile display. Performance evaluations showed that our mobile 3D quality metric monitors the degradation of quality caused by several representa…
The main focus of this document is to evaluate the performance of the existing LDR and HDR metrics on HDR video content which in turn will allow for a better understanding of how well each of these metrics work and if they can be applied in capturing, compressing, transmitting process of HDR data. To this end a series of subjective tests is performed to evaluate the quality of DML-HDR video database [1], when several different representing types of artifacts are present using a HDR display. Then, the correlation between the results from the existing LDR and HDR quality metrics and those from s…
University College Dublin（都柏林大学学院）；University of Brasilia（巴西利亚大学）
The development of audio-visual quality assessment models poses a number of challenges in order to obtain accurate predictions. One of these challenges is the modelling of the complex interaction that audio and visual stimuli have and how this interaction is interpreted by human users. The No-Reference Audio-Visual Quality Metric Based on a Deep Autoencoder (NAViDAd) deals with this problem from a machine learning perspective. The metric receives two sets of audio and video features descriptors and produces a low-dimensional set of features used to predict the audio-visual quality. A basic imp…
Reliable quality assessment of decoded point cloud geometry is essential to evaluate the compression performance of emerging point cloud coding solutions and guarantee some target quality of experience. This paper proposes a novel point cloud geometry quality assessment metric based on a generalization of the Hausdorff distance. To achieve this goal, the so-called generalized Hausdorff distance for multiple rankings is exploited to identify the best performing quality metric in terms of correlation with the MOS scores obtained from a subjective test campaign. The experimental results show that…
Point clouds (PCs) are a powerful 3D visual representation paradigm for many emerging application domains, especially virtual and augmented reality, and autonomous vehicles. However, the large amount of PC data required for highly immersive and realistic experiences requires the availability of efficient, lossy PC coding solutions are critical. Recently, two MPEG PC coding standards have been developed to address the relevant application requirements and further developments are expected in the future. In this context, the assessment of PC quality, notably for decoded PCs, is critical and asks…
Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score d…
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；Dubna State University（杜布纳国立大学）
Subjective image-quality measurement plays a critical role in the development of image-processing applications. The purpose of a visual-quality metric is to approximate the results of subjective assessment. In this regard, more and more metrics are under development, but little research has considered their limitations. This paper addresses that deficiency: we show how image preprocessing before compression can artificially increase the quality scores provided by the popular metrics DISTS, LPIPS, HaarPSI, and VIF as well as how these scores are inconsistent with subjective-quality scores. We p…
Eurecat Centre Tecnològic（Eurecat技术中心）；Satellogic Inc（Satellogic公司）
Latest advances in Super-Resolution (SR) have been tested with general purpose images such as faces, landscapes and objects, mainly unused for the task of super-resolving Earth Observation (EO) images. In this research paper, we benchmark state-of-the-art SR algorithms for distinct EO datasets using both Full-Reference and No-Reference Image Quality Assessment (IQA) metrics. We also propose a novel Quality Metric Regression Network (QMRNet) that is able to predict quality (as a No-Reference metric) by training on any property of the image (i.e. its resolution, its distortions...) and also able…
An increased interest in immersive applications has drawn attention to emerging 3D imaging representation formats, notably light fields and point clouds (PCs). Nowadays, PCs are one of the most popular 3D media formats, due to recent developments in PC acquisition, namely with new depth sensors and signal processing algorithms. To obtain high fidelity 3D representations of visual scenes a huge amount of PC data is typically acquired, which demands efficient compression solutions. As in 2D media formats, the final perceived PC quality plays an important role in the overall user experience and, …
In today's Internet, HTTP Adaptive Streaming (HAS) is the mainstream standard for video streaming, which switches the bitrate of the video content based on an Adaptive BitRate (ABR) algorithm. An effective Quality of Experience (QoE) assessment metric can provide crucial feedback to an ABR algorithm. However, predicting such real-time QoE on the client side is challenging. The QoE prediction requires high consistency with the Human Visual System (HVS), low latency, and blind assessment, which are difficult to realize together. To address this challenge, we analyzed various characteristics of H…
A localization algorithm in stochastic optical localization nanoscopy plays an important role in obtaining a high-quality image. A universal and objective metric is crucial and necessary to evaluate qualities of nanoscopy images and performances of localization algorithms. In this paper, we propose root mean square minimum distance (RMSMD) as a quality metric for localization nanoscopy images. RMSMD measures an average, local, and mutual fitness between two sets of points. Its properties common to a distance metric as well as unique to itself are presented. The ambiguity, discontinuity, and in…
