The objective of non-reference video quality assessment is to evaluate the quality of distorted video without access to reference high-definition references. In this study, we introduce an enhanced spatial perception module, pre-trained on multiple image quality assessment datasets, and a lightweight temporal fusion module to address the no-reference visual quality assessment (NR-VQA) task. This model implements Swin Transformer V2 as a local-level spatial feature extractor and fuses these multi-stage representations through a series of transformer layers. Furthermore, a temporal transformer is utilized for spatiotemporal feature fusion across the video. To accommodate compressed videos of varying bitrates, we incorporate a coarse-to-fine contrastive strategy to enrich the model's capability to discriminate features from videos of different bitrates. This is an expanded version of the one-page abstract.
University of Science and Technology of China（中国科学技术大学）
Image compression has raised widespread interest recently due to its significant importance for multimedia storage and transmission. Meanwhile, a reliable image quality assessment (IQA) for compressed images can not only help to verify the performance of various compression algorithms but also help to guide the compression optimization in turn. In this paper, we design a full-reference image quality assessment metric SwinIQA to measure the perceptual quality of compressed images in a learned Swin distance space. It is known that the compression artifacts are usually non-uniformly distributed w…
Measuring the perceptual quality of images automatically is an essential task in the area of computer vision, as degradations on image quality can exist in many processes from image acquisition, transmission to enhancing. Many Image Quality Assessment(IQA) algorithms have been designed to tackle this problem. However, it still remains un settled due to the various types of image distortions and the lack of large-scale human-rated datasets. In this paper, we propose a novel algorithm based on the Swin Transformer [31] with fused features from multiple stages, which aggregates information from b…
Super-resolution, which aims to reconstruct high-resolution images from low-resolution images, has drawn considerable attention and has been intensively studied in computer vision and remote sensing communities. The super-resolution technology is especially beneficial for Unmanned Aerial Vehicles (UAV), as the amount and resolution of images captured by UAV are highly limited by physical constraints such as flight altitude and load capacity. In the wake of the successful application of deep learning methods in the super-resolution task, in recent years, a series of super-resolution algorithms …
REQA: Coarse-to-fine Assessment of Image Quality to Alleviate the Range Effect
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Huawei Technologies Co., Ltd.（华为技术有限公司）；The Hong Kong Polytechnic University（香港理工大学）
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；ADAPT SFI Research Centre（ADAPT SFI研究中心）；University College Dublin（都柏林大学）；Nanyang Technological University Singapore（新加坡南洋理工大学）；National University of Singapore（新加坡国立大学）
Image quality assessment is critical to control and maintain the perceived quality of visual content. Both subjective and objective evaluations can be utilised, however, subjective image quality assessment is currently considered the most reliable approach. Databases containing distorted images and mean opinion scores are needed in the field of atmospheric research with a view to improve the current state-of-the-art methodologies. In this paper, we focus on using ground-based sky camera images to understand the atmospheric events. We present a new image quality assessment dataset containing or…
NORCE Norwegian Research Centre（NORCE挪威研究中心）；Hong Kong University of Science and Technology（香港科技大学）
Visual (image, video) quality assessments can be modelled by visual features in different domains, e.g., spatial, frequency, and temporal domains. Perceptual mechanisms in the human visual system (HVS) play a crucial role in generation of quality perception. This paper proposes a general framework for no-reference visual quality assessment using efficient windowed transformer architectures. A lightweight module for multi-stage channel attention is integrated into Swin (shifted window) Transformer. Such module can represent appropriate perceptual mechanisms in image quality assessment (IQA) to …
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
No-Reference Image Quality Assessment (NR-IQA) aims to assess the perceptual quality of images in accordance with human subjective perception. Unfortunately, existing NR-IQA methods are far from meeting the needs of predicting accurate quality scores on GAN-based distortion images. To this end, we propose Multi-dimension Attention Network for no-reference Image Quality Assessment (MANIQA) to improve the performance on GAN-based distortion. We firstly extract features via ViT, then to strengthen global and local interactions, we propose the Transposed Attention Block (TAB) and the Scale Swin Tr…
Applications based on synergistic integration of optical imagery and LiDAR data are receiving a growing interest from the remote sensing community. However, a misaligned integration between these datasets may fail to fully profit the potential of both sensors. In this regard, an optimum fusion of optical imagery and LiDAR data requires an accurate registration. This is a complex problem since a versatile solution is still missing, especially when considering the context where data are collected at different times, from different platforms, under different acquisition configurations. This paper…
Recent years have witnessed the rapid development of image storage and transmission systems, in which image compression plays an important role. Generally speaking, image compression algorithms are developed to ensure good visual quality at limited bit rates. However, due to the different compression optimization methods, the compressed images may have different levels of quality, which needs to be evaluated quantificationally. Nowadays, the mainstream full-reference (FR) metrics are effective to predict the quality of compressed images at coarse-grained levels (the bit rates differences of co…
Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models
South China Normal University（华南师范大学）；Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Shenzhen Academy of Inspection and Quarantine（深圳检验检疫局）；Shenzhen University（深圳大学）
DEFNet: Multitasks-based Deep Evidential Fusion Network for Blind Image Quality Assessment
Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education（高可信软件技术重点实验室（北京大学）教育部）；School of Computer Science, Peking University（计算机学院（北京大学））
MS-IQA: A Multi-Scale Feature Fusion Network for PET/CT Image Quality Assessment
Harbin Institute of Technology（哈尔滨工业大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；Harbin Medical University（哈尔滨医科大学）；Nanyang Technological University（南洋理工大学）；Sichuan University of Science and Engineering（四川理工学院）
BiRQA: Bidirectional Robust Quality Assessment for Images
MSU Institute for Artificial Intelligence（莫斯科人工智能研究所）；Lomonosov Moscow State University（洛蒙诺索夫莫斯科国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（可信人工智能俄罗斯科学院ISP研究中心）；Innopolis University（英诺波利斯大学）
AI-Generated Image Quality Assessment Based on Task-Specific Prompt and Multi-Granularity Similarity
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州研究院）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Istanbul Technical University（伊斯坦布尔技术大学）；Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery Training and Research Hospital（穆罕默德·阿基夫·埃尔索伊胸心血管外科培训研究医院）；Acibadem University（阿哲巴登大学）；King’s College London（伦敦国王学院）
To maintain a standard in a medical imaging study, images should have necessary image quality for potential diagnostic use. Although CNN-based approaches are used to assess the image quality, their performance can still be improved in terms of accuracy. In this work, we approach this problem by using Swin Transformer, which improves the poor-quality image classification performance that causes the degradation in medical image quality. We test our approach on Foreign Object Classification problem on Chest X-Rays (Object-CXR) and Left Ventricular Outflow Tract Classification problem on Cardiac M…
A Deep Learning approach for Depressive Symptoms assessment in Parkinson's disease patients using facial videos
Foundation for Research and Technology Hellas (FORTH)（希腊研究与技术基金会）；University of Crete（克里特大学）；University Hospital of Heraklion（伊拉克利翁大学医院）；Patras University Hospital（帕特雷大学医院）；University of Ioannina（约阿尼纳大学）
Korea University（高丽大学）；Microsoft Research Asia（微软亚洲研究院）
This paper presents a novel cost aggregation network, called Volumetric Aggregation with Transformers (VAT), for few-shot segmentation. The use of transformers can benefit correlation map aggregation through self-attention over a global receptive field. However, the tokenization of a correlation map for transformer processing can be detrimental, because the discontinuity at token boundaries reduces the local context available near the token edges and decreases inductive bias. To address this problem, we propose a 4D Convolutional Swin Transformer, where a high-dimensional Swin Transformer is p…
Digital humans are attracting more and more research interest during the last decade, the generation, representation, rendering, and animation of which have been put into large amounts of effort. However, the quality assessment of digital humans has fallen behind. Therefore, to tackle the challenge of digital human quality assessment issues, we propose the first large-scale quality assessment database for three-dimensional (3D) scanned digital human heads (DHHs). The constructed database consists of 55 reference DHHs and 1,540 distorted DHHs along with the subjective perceptual ratings. Then, …
Sano Centre for Computational Medicine（萨诺计算医学中心）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Cardiac Magnetic Resonance Imaging is commonly used for the assessment of the cardiac anatomy and function. The delineations of left and right ventricle blood pools and left ventricular myocardium are important for the diagnosis of cardiac diseases. Unfortunately, the movement of a patient during the CMR acquisition procedure may result in motion artifacts appearing in the final image. Such artifacts decrease the diagnostic quality of CMR images and force redoing of the procedure. In this paper, we present a Multi-task Swin UNEt TRansformer network for simultaneous solving of two tasks in the …
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Shenzhen Medical Biometrics Perception and Analysis Engineering Laboratory（深圳市医学生物特征感知与分析工程实验室）；Bio-Computing Research Center（生物计算研究中心）；Shenzhen Key Laboratory of Visual Object Detection and Recognition（深圳市视觉目标检测与识别重点实验室）
Automatic medical image segmentation has made great progress benefit from the development of deep learning. However, most existing methods are based on convolutional neural networks (CNNs), which fail to build long-range dependencies and global context connections due to the limitation of receptive field in convolution operation. Inspired by the success of Transformer in modeling the long-range contextual information, some researchers have expended considerable efforts in designing the robust variants of Transformer-based U-Net. Moreover, the patch division used in vision transformers usually …
FLASHSWIN: Unlocking Large Windows and Dense Tokens in Swin Vision Transformers with Memory Efficient Attention
Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与影像研究所）；Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；RNET Technologies（RNET技术公司）
UPDA: Unsupervised Progressive Domain Adaptation for No-Reference Point Cloud Quality Assessment
School of Automation, Nanjing University of Science and Technology（自动化学院，南京理工大学）；National Key Laboratory of Information Systems Engineering（信息系统工程国家重点实验室）；School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）
University of Rochester（罗切斯特大学）；Houston Methodist（休斯顿卫理公会医院）；University of Rochester Medical Center（罗切斯特大学医学中心）
In this paper, we investigated whether we can 1) detect participants with ataxia-specific gait characteristics (risk-prediction), and 2) assess severity of ataxia from gait (severity-assessment) using computer vision. We created a dataset of 155 videos from 89 participants, 24 controls and 65 diagnosed with (or are pre-manifest) spinocerebellar ataxias (SCAs), performing the gait task of the Scale for the Assessment and Rating of Ataxia (SARA) from 11 medical sites located in 8 different states across the United States. We develop a computer vision pipeline to detect, track, and separate out t…
A Fully Automated Pipeline Using Swin Transformers for Deep Learning-Based Blood Segmentation on Head CT Scans After Aneurysmal Subarachnoid Hemorrhage
GeoFormer: A Lightweight Swin Transformer for Joint Building Height and Footprint Estimation from Sentinel Imagery
Department of Civil, Architectural and Environmental System Engineering, Sungkyunkwan University（成均馆大学土木、建筑与环境系统工程系）；School of Geography, University of Leeds（利兹大学地理学院）；Interdisciplinary Program for Crisis, Disaster and Risk Management, Sungkyunkwan University（成均馆大学危机、灾害与风险管理跨学科项目）
Qingdao University（青岛大学）；New York University（纽约大学）；Shandong University（山东大学）；De Montfort University（德蒙福特大学）
With the wide applications of colored point cloud in many fields, point cloud perceptual quality assessment plays a vital role in the visual communication systems owing to the existence of quality degradations introduced in various stages. However, the existing point cloud quality assessments ignore the mechanism of human visual system (HVS) which has an important impact on the accuracy of the perceptual quality assessment. In this paper, a progressive knowledge transfer based on human visual perception mechanism for perceptual quality assessment of point clouds (PKT-PCQA) is proposed. The PKT…
Uncertainty-Guided Coarse-to-Fine Tumor Segmentation with Anatomy-Aware Post-Processing
Department of Computer Science University of Central Florida Orlando, FL, USA（计算机科学系 佛罗里达中央大学 或兰多市）；Department of Radiology Northwestern University Chicago, IL, USA（放射学系 北western大学 芝加哥市）
Granular Computing-driven SAM: From Coarse-to-Fine Guidance for Prompt-Free Segmentation
School of Computer Science and Software Engineering（计算机科学与软件工程学院）；Southwest Petroleum University（西南石油大学）；School of Computing and Artificial Intelligence（计算与人工智能学院）；Southwest Jiaotong University（西南交通大学）；Southwestern University of Finance and Economics（西南财经大学）；College of Computer Science（计算机科学学院）；Sichuan University（四川大学）
