Image quality assessment (IQA) is an important element of a broad spectrum of applications ranging from automatic video streaming to display technology. Furthermore, the measurement of image quality requires a balanced investigation of image content and features. Our proposed approach extracts visual features by attaching global average pooling (GAP) layers to multiple Inception modules of on an ImageNet database pretrained convolutional neural network (CNN). In contrast to previous methods, we do not take patches from the input image. Instead, the input image is treated as a whole and is run through a pretrained CNN body to extract resolution-independent, multi-level deep features. As a consequence, our method can be easily generalized to any input image size and pretrained CNNs. Thus, we present a detailed parameter study with respect to the CNN base architectures and the effectiveness of different deep features. We demonstrate that our best proposal - called MultiGAP-NRIQA - is able to provide state-of-the-art results on three benchmark IQA databases. Furthermore, these results were also confirmed in a cross database test using the LIVE In the Wild Image Quality Challenge database.
We propose an effective deep learning approach to aesthetics quality assessment that relies on a new type of pre-trained features, and apply it to the AVA data set, the currently largest aesthetics database. While previous approaches miss some of the information in the original images, due to taking small crops, down-scaling or warping the originals during training, we propose the first method that efficiently supports full resolution images as an input, and can be trained on variable input sizes. This allows us to significantly improve upon the state of the art, increasing the Spearman rank-o…
In this work, we present an empirical study of DCGANs, including hyperparameter heuristics and image quality assessment, as a way to address the scarcity of datasets to investigate fetal head ultrasound. We present experiments to show the impact of different image resolutions, epochs, dataset size input, and learning rates for quality image assessment on four metrics: mutual information (MI), Fréchet inception distance (FID), peak-signal-to-noise ratio (PSNR), and local binary pattern vector (LBPv). The results show that FID and LBPv have stronger relationship with clinical image quality score…
Wuhan University（武汉大学）；New York University（纽约大学）；University of Waterloo（滑铁卢大学）
We propose a deep bilinear model for blind image quality assessment (BIQA) that handles both synthetic and authentic distortions. Our model consists of two convolutional neural networks (CNN), each of which specializes in one distortion scenario. For synthetic distortions, we pre-train a CNN to classify image distortion type and level, where we enjoy large-scale training data. For authentic distortions, we adopt a pre-trained CNN for image classification. The features from the two CNNs are pooled bilinearly into a unified representation for final quality prediction. We then fine-tune the entir…
University of Konstanz（康斯坦茨大学）；Institute for Computer Science and Control (SZTAKI)（计算机科学与控制研究所（SZTAKI））
Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 m…
Generative adversarial networks or GANs are a type of generative modeling framework. GANs involve a pair of neural networks engaged in a competition in iteratively creating fake data, indistinguishable from the real data. One notable application of GANs is developing fake human faces, also known as "deep fakes," due to the deep learning algorithms at the core of the GAN framework. Measuring the quality of the generated images is inherently subjective but attempts to objectify quality using standardized metrics have been made. One example of objective metrics is the Frechet Inception Distance (…
Synthetic generation of three-dimensional cell models from histopathological images aims to enhance understanding of cell mutation, and progression of cancer, necessary for clinical assessment and optimal treatment. Classical reconstruction algorithms based on image registration of consecutive slides of stained tissues are prone to errors and often not suitable for the training of three-dimensional segmentation algorithms. We propose a novel framework to generate synthetic three-dimensional histological models based on a generator-discriminator pattern optimizing constrained features that cons…
The University of Tokyo（东京大学）；RIKEN（理化学研究所）；Dalian University of Technology（大连理工大学）；AIST（产业技术综合研究所）
Automated video-based assessment of surgical skills is a promising task in assisting young surgical trainees, especially in poor-resource areas. Existing works often resort to a CNN-LSTM joint framework that models long-term relationships by LSTMs on spatially pooled short-term CNN features. However, this practice would inevitably neglect the difference among semantic concepts such as tools, tissues, and background in the spatial dimension, impeding the subsequent temporal relationship modeling. In this paper, we propose a novel skill assessment framework, Video Semantic Aggregation (ViSA), wh…
Beijing Tongren Hospital（北京同仁医院）；University of Pittsburgh（匹兹堡大学）；Shaanxi Provincial People's Hospital（陕西省人民医院）
Objective: To validate and compare the performance of eight available deep learning architectures in grading the severity of glaucoma based on color fundus images. Materials and Methods: We retrospectively collected a dataset of 5978 fundus images and their glaucoma severities were annotated by the consensus of two experienced ophthalmologists. We preprocessed the images to generate global and local regions of interest (ROIs), namely the global field-of-view images and the local disc region images. We then divided the generated images into three independent sub-groups for training, validation,…
Fudan University（复旦大学）；Shanghai General Hospital, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属上海市第一人民医院）
The ultra-wide optical coherence tomography angiography (OCTA) has become an important imaging modality in diabetic retinopathy (DR) diagnosis. However, there are few researches focusing on automatic DR analysis using ultra-wide OCTA. In this paper, we present novel and practical deep-learning solutions based on ultra-wide OCTA for the Diabetic Retinopathy Analysis Challenge (DRAC). In the segmentation of DR lesions task, we utilize UNet and UNet++ to segment three lesions with strong data augmentation and model ensemble. In the image quality assessment task, we create an ensemble of Inception…
Ningbo University（宁波大学）；University of Waterloo（滑铁卢大学）；Shanghai Jiao Tong University（上海交通大学）
Blind image quality assessment (BIQA), which aims to accurately predict the image quality without any pristine reference information, has been extensively concerned in the past decades. Especially, with the help of deep neural networks, great progress has been achieved. However, it remains less investigated on BIQA for night-time images (NTIs) which usually suffers from complicated authentic distortions such as reduced visibility, low contrast, additive noises, and color distortions. These diverse authentic degradations particularly challenges the design of effective deep neural network for bl…
National Engineering Research Center for Ophthalmology（国家眼科工程研究中心）；Beijing Institute of Ophthalmology（北京眼科研究所）；Beijing Tongren Hospital（北京同仁医院）；Capital Medical University（首都医科大学）；University of Pittsburgh（匹兹堡大学）
To investigate whether and to what extent central serous chorioretinopathy (CSC) depicted on color fundus photographs can be assessed using deep learning technology. We collected a total of 2,504 fundus images acquired on different subjects. We verified the CSC status of these images using their corresponding optical coherence tomography (OCT) images. A total of 1,329 images depicted CSC. These images were preprocessed and normalized. This resulting dataset was randomly split into three parts in the ratio of 8:1:1 respectively for training, validation, and testing purposes. We used the deep le…
University of Southampton（南安普顿大学）；University Hospital Southampton NHS Foundation Trust（南安普顿大学医院NHS基金会信托）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
A desire to achieve large medical imaging datasets keeps increasing as machine learning algorithms, parallel computing, and hardware technology evolve. Accordingly, there is a growing demand in pooling data from multiple clinical and academic institutes to enable large-scale clinical or translational research studies. Magnetic resonance imaging (MRI) is a frequently used, non-invasive imaging modality. However, constructing a big MRI data repository has multiple challenges related to privacy, data size, DICOM format, logistics, and non-standardized images. Not only building the data repository…
Massachusetts General Hospital, Harvard Medical School（哈佛医学院麻省总医院）；Harvard Medical School（哈佛医学院）
A critical factor that influences the success of an in-vitro fertilization (IVF) procedure is the quality of the transferred embryo. Embryo morphology assessments, conventionally performed through manual microscopic analysis suffer from disparities in practice, selection criteria, and subjectivity due to the experience of the embryologist. Convolutional neural networks (CNNs) are powerful, promising algorithms with significant potential for accurate classifications across many object categories. Network architectures and hyper-parameters affect the efficiency of CNNs for any given task. Here, …
Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality
Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；Department of Internal Medicine, Division of Epidemiology, University of Utah（犹他大学内科学系流行病学部门）；Department of Pathology, University of Utah（犹他大学病理学系）；ARUP Laboratories（ARUP实验室）
An edge detection-based deep learning approach for tear meniscus height measurement
College of Mathematical Medicine, Zhejiang Normal University（浙江师范大学数理医学院）；National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University（温州医科大学附属眼视光医院国家眼部疾病临床医学研究中心）；Puyang Big Data and Artificial Intelligence Institute（濮阳大数据与人工智能研究院）
Video quality assessment (VQA) methods focus on particular degradation types, usually artificially induced on a small set of reference videos. Hence, most traditional VQA methods under-perform in-the-wild. Deep learning approaches have had limited success due to the small size and diversity of existing VQA datasets, either artificial or authentically distorted. We introduce a new in-the-wild VQA dataset that is substantially larger and diverse: KonVid-150k. It consists of a coarsely annotated set of 153,841 videos having five quality ratings each, and 1,596 videos with a minimum of 89 ratings …
Federated Multi-Task Learning for Bladder Tumor Segmentation and MIBC Classification Using a Hybrid CNN-Transformer Architecture
Indian Institute of Information Technology Allahabad（印度阿拉巴德信息技术学院）；Manipal University Jaipur（斋浦尔马尼帕尔大学）；Norwegian University of Science and Technology（挪威科技大学）
Origin Health India（印度Origin Health）；Bangalore Fetal Medicine Center（班加罗尔胎儿医学中心）；Prime Imaging and Prenatal Diagnostics（Prime影像与产前诊断中心）；Kasturba Medical College, Manipal Academy of Higher Education（马尼帕尔高等教育学院卡斯特尔巴医学院）；Origin Health；Universiti Malaya（马来亚大学）；Aarthi Scans and Labs（Aarthi扫描与实验室）
Quality assessment of prenatal ultrasonography is essential for the screening of fetal central nervous system (CNS) anomalies. The interpretation of fetal brain structures is highly subjective, expertise-driven, and requires years of training experience, limiting quality prenatal care for all pregnant mothers. With recent advancement in Artificial Intelligence (AI), specifically deep learning (DL), assistance in precise anatomy identification through semantic segmentation essential for the reliable assessment of growth and neurodevelopment, and detection of structural abnormalities have been p…
HP-GAN: Harnessing pretrained networks for GAN improvement with FakeTwins and discriminator consistency
School of Electrical and Electronic Engineering, Yonsei University, Seoul, Republic of Korea（电气电子工程学院，延世大学，首尔，大韩民国）；Robotics Institute, Korea Institute of Science and Technology, Seoul, Republic of Korea（机器人研究所，韩国科学技术院，首尔，大韩民国）；Department of Oral and Maxillofacial Radiology, Yonsei University College of Dentistry, Seoul, Republic of Korea（口腔和颌面放射科，延世大学牙科学院，首尔，大韩民国）
Evaluation of Activated Sludge Settling Characteristics from Microscopy Images with Deep Convolutional Neural Networks and Transfer Learning
IVL Swedish Environmental Research Institute（瑞典环境科学研究院）；Ghent University（根特大学）；University of São Paulo（圣保罗大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；CAPTURE, Centre for Advanced Process Technology for Urban Resource Recovery（城市资源回收先进过程技术中心）；Pantarein Water（Pantarein Water 公司）
Universitat de Barcelona（巴塞罗那大学）；University of Arkansas for Medical Sciences（阿肯色医科大学）
Synthetic data generated by generative models can enhance the performance and capabilities of data-hungry deep learning models in medical imaging. However, there is (1) limited availability of (synthetic) datasets and (2) generative models are complex to train, which hinders their adoption in research and clinical applications. To reduce this entry barrier, we propose medigan, a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library. medigan allows researchers and developers to create, increase, and domain-adapt their training data in jus…
R2MF-Net: A Recurrent Residual Multi-Path Fusion Network for Robust Multi-directional Spine X-ray Segmentation
School of Information Science & Engineering, Shandong Normal University（信息科学与工程学院，山东师范大学）；Tajik State University of Law, Business and Politics（塔吉克国立法律、商业与政治大学）
Multi-Sensor Diffusion-Driven Optical Image Translation for Large-Scale Applications
Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院）；Aeronautics Institute of Technology (ITA)（航空技术学院）；Universidade Federal de Santa Catarina (UFSC)（圣卡塔琳娜联邦大学）
针对大规模多传感器光学影像域转换中的斑块不一致问题，提出新型正反向扩散超分方法，在 Sentinel-II 与 Planet Dove 数据上显著优于 DDIM 等方法。
