The performance of objective image quality assessment (IQA) models has been evaluated primarily by comparing model predictions to human quality judgments. Perceptual datasets gathered for this purpose have provided useful benchmarks for improving IQA methods, but their heavy use creates a risk of overfitting. Here, we perform a large-scale comparison of IQA models in terms of their use as objectives for the optimization of image processing algorithms. Specifically, we use eleven full-reference IQA models to train deep neural networks for four low-level vision tasks: denoising, deblurring, super-resolution, and compression. Subjective testing on the optimized images allows us to rank the competing models in terms of their perceptual performance, elucidate their relative advantages and disadvantages in these tasks, and propose a set of desirable properties for incorporation into future IQA models.
The quality of visual input is very important for both human and machine perception. Consequently many processing techniques exist that deal with different distortions. Usually image processing is applied freely and lacks redundancy regarding safety. We propose a novel image comparison method called the Combined Distances of Contrast Distributions (CD2) to protect against errors that arise during processing. Based on the distribution of image contrasts a new reduced-reference image quality assessment (IQA) method is introduced. By combining various distance functions excellent performance on I…
Space and Naval Warfare Systems Center Pacific（太平洋空间与海战系统中心）
Traditional metrics for evaluating the efficacy of image processing techniques do not lend themselves to understanding the capabilities and limitations of modern image processing methods - particularly those enabled by deep learning. When applying image processing in engineering solutions, a scientist or engineer has a need to justify their design decisions with clear metrics. By applying blind/referenceless image spatial quality (BRISQUE), Structural SIMilarity (SSIM) index scores, and Peak signal-to-noise ratio (PSNR) to images before and after image processing, we can quantify quality impro…
In subjective full-reference image quality assessment, differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcome these problems. In addition, PC is more sensitive than DCR, and it c…
Expert-Agnostic Ultrasound Image Quality Assessment using Deep Variational Clustering
All India Institute of Medical Sciences (AIIMS)（全印医学科学学院）；Purdue University (PU)（普渡大学）；Indian Institute of Technology (IIT)（印度理工学院）；Berea College（伯里亚学院）
Image quality assessment (IQA) is traditionally classified into full-reference (FR) IQA and no-reference (NR) IQA according to whether the original image is required. Although NR-IQA is widely used in practical applications, room for improvement still remains because of the lack of the reference image. Inspired by the fact that in many applications, such as parameter selection, a series of distorted images are available, the authors propose a novel comparison-based image quality assessment (C-IQA) method. The new comparison-based framework parallels FR-IQA by requiring two input images, and re…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Google Inc.（谷歌公司）
Video and image quality assessment has long been projected as a regression problem, which requires predicting a continuous quality score given an input stimulus. However, recent efforts have shown that accurate quality score regression on real-world user-generated content (UGC) is a very challenging task. To make the problem more tractable, we propose two new methods - binary, and ordinal classification - as alternatives to evaluate and compare no-reference quality models at coarser levels. Moreover, the proposed new tasks convey more practical meaning on perceptually optimized UGC transcoding…
University of Amsterdam（阿姆斯特丹大学）；Sony AI（索尼AI）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Image quality assessment (IQA) forms a natural and often straightforward undertaking for humans, yet effective automation of the task remains highly challenging. Recent metrics from the deep learning community commonly compare image pairs during training to improve upon traditional metrics such as PSNR or SSIM. However, current comparisons ignore the fact that image content affects quality assessment as comparisons only occur between images of similar content. This restricts the diversity and number of image pairs that the model is exposed to during training. In this paper, we strive to enrich…
One of the big challenges of the industry today is how to produce quality products, one of which is in the non-woven industry. The improvement of the evaluation and quality control processes of non woven production has been widely developed to support the improvement of the quality of production. The use of information and computational technology has now been widely applied to the quality control process of textile material production, one of which is the use of image processing technology in the evaluation process of non-woven materials. This paper will explain various methods of applying im…
One of the big challenges of the industry today is how to produce quality products, one of which is in the knit fabric industry. The improvement of the evaluation and quality control processes of non woven production has been widely developed to support the improvement of the quality of production. The use of information and computational technology is now widely applied to the quality control process of textile material production, one of which is the use of image processing technology in the evaluation process of knit fabric. This chapter will explain various methods of applying image proces…
To effectively evaluate subjective visual quality in weakly-controlled environments, we propose an Adaptive Paired Comparison method based on particle filtering. As our approach requires each sample to be rated only once, the test time compared to regular paired comparison can be reduced. The method works with non-experts and improves reliability compared to MOS and DS-MOS methods.
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）
Quality control (QC) in medical image analysis is time-consuming and laborious, leading to increased interest in automated methods. However, what is deemed suitable quality for algorithmic processing may be different from human-perceived measures of visual quality. In this work, we pose MR image quality assessment from an image reconstruction perspective. We train Bayesian CNNs using a heteroscedastic uncertainty model to recover clean images from noisy data, providing measures of uncertainty over the predictions. This framework enables us to divide data corruption into learnable and non-learn…
Bristol Vision Institute（布里斯托尔视觉研究所）；University of Bristol（布里斯托尔大学）
In recent years, video compression techniques have been significantly challenged by the rapidly increased demands associated with high quality and immersive video content. Among various compression tools, post-processing can be applied on reconstructed video content to mitigate visible compression artefacts and to enhance overall perceptual quality. Inspired by advances in deep learning, we propose a new CNN-based post-processing approach, which has been integrated with two state-of-the-art coding standards, VVC and AV1. The results show consistent coding gains on all tested sequences at vario…
University of Westminster（威斯敏斯特大学）；Calico Life Sciences LLC（Calico生命科学有限责任公司）
An end-to-end image analysis pipeline is presented for the abdominal MRI protocol used in the UK Biobank on the first 38,971 participants. Emphasis is on the processing steps necessary to ensure a high-level of data quality and consistency is produced in order to prepare the datasets for downstream quantitative analysis, such as segmentation and parameter estimation. Quality control procedures have been incorporated to detect and, where possible, correct issues in the raw data. Detection of fat-water swaps in the Dixon series is performed by a deep learning model and corrected automatically. B…
Prince Sultan University（苏丹王子大学）；Polytechnic Institute of Porto（波尔图理工学院）；University of Carthage（迦太基大学）
In this paper, we address the problem of car detection from aerial images using Convolutional Neural Networks (CNN). This problem presents additional challenges as compared to car (or any object) detection from ground images because features of vehicles from aerial images are more difficult to discern. To investigate this issue, we assess the performance of two state-of-the-art CNN algorithms, namely Faster R-CNN, which is the most popular region-based algorithm, and YOLOv3, which is known to be the fastest detection algorithm. We analyze two datasets with different characteristics to check th…
Fast Adversarial CNN-based Perturbation Attack on No-Reference Image- and Video-Quality Metrics
Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）
A problem often arising in video communication is the reconstruction of missing or distorted areas in a video sequence. Such holes of unavailable pixels may be caused for example by transmission errors of coded video data or undesired objects like logos. In order to close the holes given neighboring available content, a signal extrapolation has to be performed. The best quality can be achieved, if spatial as well as temporal information is used for the reconstruction. However, the question always is in which order to process the extrapolation to obtain the best result. In this paper, an optimi…
VVC is the next generation video coding standard, offering coding capability beyond HEVC standard. The high computational complexity of the latest video coding standards requires high-level parallelism techniques, in order to achieve real-time and low latency encoding and decoding. HEVC and VVC include tile grid partitioning that allows to process simultaneously rectangular regions of a frame with independent threads. The tile grid may be further partitioned into a horizontal sub-grid of Rectangular Slices (RSs), increasing the partitioning flexibility. The dynamic Tile and Rectangular Slice (…
The rapid development of multimedia and communication technology has resulted in an urgent need for high-quality video streaming. However, robust video streaming under fluctuating network conditions and heterogeneous client computing capabilities remains a challenge. In this paper, we consider an enhancement-enabled video streaming network under a time-varying wireless network and limited computation capacity. "Enhancement" means that the client can improve the quality of the downloaded video segments via image processing modules. We aim to design a joint bitrate adaptation and client-side enh…
Quality is one of the important things to be maintained in a weaving industry. Along with the times, technological developments in the field of image processing and computing have changed the old method of visual evaluation of woven fabric to be better. This chapter will explain the implementation of image and computational processing techniques on woven fabric. Keywords: woven fabric, digital image processing.
Different tasks in the computational pipeline of single-particle cryo-electron microscopy (cryo-EM) require enhancing the quality of the highly noisy raw images. To this end, we develop an efficient algorithm for signal enhancement of cryo-EM images. The enhanced images can be used for a variety of downstream tasks, such as 2-D classification, removing uninformative images, constructing {ab initio} models, generating templates for particle picking, providing a quick assessment of the data set, dimensionality reduction, and symmetry detection. The algorithm includes built-in quality measures to…
The comparison between two approaches, JPEG and Compressive Sensing, is done in the paper. The approaches are compared in terms of image compression. Comparison is done by measuring the image quality versus number of samples used for image recovering. Images are visually compared. Also, numerical quality value, PSNR, is calculated and compared for the two approaches. It is shown that images, recovered by using the Compressive Sensing approach, have higher PSNR values compared to the images under JPEG compression. Difference is larger in grayscale images with small number of details, like e.g. …
