The spatial and spectral degradation functions are critical to hyper- and multi-spectral image fusion. However, few work has been payed on the estimation of the degradation functions. To learn the spatial response function and the point spread function from the image pairs to be fused, we propose a Dirichlet network, where both functions are properly constrained. Specifically, the spatial response function is constrained with positivity, while the Dirichlet distribution along with a total variation is imposed on the point spread function. To the best of our knowledge, the neural netwrok and the Dirichlet regularization are exclusively investigated, for the first time, to estimate the degradation functions. Both image degradation and fusion experiments demonstrate the effectiveness and superiority of the proposed Dirichlet network.
Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations
VCIP, College of Computer Science, Nankai University, Tianjin, 300350, China（VCIP，计算机科学学院，南开大学，天津，300350，中国）；School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University, Shenzhen 518107, P.R. China（网络安全科学与技术学院，深圳校区，中山大学，深圳 518107，中华人民共和国中国）；College of Artificial Intelligence, Nankai University, Tianjin, 300350, China（人工智能学院，南开大学，天津，300350，中国）；Laboratory of Science and Technology on Integrated Logistics Support, National University of Defense Technology, Changsha 410073, China（集成物流支持科学技术实验室，国防科技大学，长沙 410073，中国）
Physics-Inspired Degradation Models for Hyperspectral Image Fusion
Beijing Institute of Technology（北京理工大学）；Hunan University（湖南大学）；University of Science and Technology of China（中国科学技术大学）；Beijing Normal University（北京师范大学）
Michigan State University（密歇根州立大学）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regio…
Degradation Frequency Curve: An Explicit Frequency-Quantified Representation for All-in-One Image Restoration
College of Electronic Science, National University of Defense Technology（国防科技大学电子科学学院）；College of Electronic Engineering, Beihang University（北航电子工程学院）；VCIP, School of Computer Science, Nankai University（南开大学计算机科学学院）
Degradation-Aware Metric Prompting for Hyperspectral Image Restoration
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China（北京理工大学计算机科学与技术学院）；School of Computer Science, Wuhan University, Wuhan, Hubei, China（武汉大学计算机学院）；Zhongguancun Academy, Beijing, China（中关村学院）
Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation
University of Trento（特伦托大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Würzburg（乌尔姆大学）；ETH Zürich（苏黎世联邦理工学院）；Taiyuan University of Technology（太原理工大学）；University of California, Merced（加州大学默塞德分校）
Image super-resolution (SR) research has witnessed impressive progress thanks to the advance of convolutional neural networks (CNNs) in recent years. However, most existing SR methods are non-blind and assume that degradation has a single fixed and known distribution (e.g., bicubic) which struggle while handling degradation in real-world data that usually follows a multi-modal, spatially variant, and unknown distribution. The recent blind SR studies address this issue via degradation estimation, but they do not generalize well to multi-source degradation and cannot handle spatially variant deg…
To overcome inherent hardware limitations of hyperspectral imaging systems with respect to their spatial resolution, fusion-based hyperspectral image (HSI) super-resolution is attracting increasing attention. This technique aims to fuse a low-resolution (LR) HSI and a conventional high-resolution (HR) RGB image in order to obtain an HR HSI. Recently, deep learning architectures have been used to address the HSI super-resolution problem and have achieved remarkable performance. However, they ignore the degradation model even though this model has a clear physical interpretation and may contribu…
Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration
Southern University of Science and Technology（南方科技大学）；Shenzhen University（深圳大学）；Shenzhen University of Advanced Technology（深圳大学先进技术学院）；Xi’an Jiaotong University（西安交通大学）
Image restoration is the task of recovering a clean image from a degraded version. In most cases, the degradation is spatially varying, and it requires the restoration network to both localize and restore the affected regions. In this paper, we present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images affected by practically occurring artifacts such as blur, rain-streaks. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration, unlike existing methods which directly learn a …
PG-ControlNet: A Physics-Guided ControlNet for Generative Spatially Varying Image Deblurring
1 Computer Science Department, University of Rochester；2 Goergen Institute for Data Science \& AIS, University of Rochester；Computer Engineering Department, University of Rochester
Department of Computing Science University of Alberta（阿尔伯塔大学计算机科学系）
Deep convolution neural networks (CNNs) play a critical role in single image super-resolution (SISR) since the amazing improvement of high performance computing. However, most of the super-resolution (SR) methods only focus on recovering bicubic degradation. Reconstructing high-resolution (HR) images from randomly blurred and noisy low-resolution (LR) images is still a challenging problem. In this paper, we propose a novel Spatial Context Hallucination Network (SCHN) for blind super-resolution without knowing the degradation kernel. We find that when the blur kernel is unknown, separate deblur…
TIDE: Two-Stage Inverse Degradation Estimation with Guided Prior Disentanglement for Underwater Image Restoration
Mohamed bin Zayed University of AI（莫扎德·本·泽亚德人工智能大学）；University of Amsterdam（阿姆斯特丹大学）；University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）；Nagasaki University（长崎大学）
University of Science and Technology of China（中国科学技术大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
Light field image quality assessment (LFI-QA) is a significant and challenging research problem. It helps to better guide light field acquisition, processing and applications. However, only a few objective models have been proposed and none of them completely consider intrinsic factors affecting the LFI quality. In this paper, we propose a No-Reference Light Field image Quality Assessment (NR-LFQA) scheme, where the main idea is to quantify the LFI quality degradation through evaluating the spatial quality and angular consistency. We first measure the spatial quality deterioration by capturing…
Deep Convolutional Neural Networks (CNNs) have achieved remarkable results on Single Image Super-Resolution (SISR). Despite considering only a single degradation, recent studies also include multiple degrading effects to better reflect real-world cases. However, most of the works assume a fixed combination of degrading effects, or even train an individual network for different combinations. Instead, a more practical approach is to train a single network for wide-ranging and variational degradations. To fulfill this requirement, this paper proposes a unified network to accommodate the variation…
CUHK（香港中文大学）；NVIDIA AI Technology Center（英伟达人工智能技术中心）；SenseTime Research（商汤科技研究院）；Centre for Perceptual and Interactive Intelligence Limited（感知与交互智能中心有限公司）；Xidian University（西安电子科技大学）
In various learning-based image restoration tasks, such as image denoising and image super-resolution, the degradation representations were widely used to model the degradation process and handle complicated degradation patterns. However, they are less explored in learning-based image deblurring as blur kernel estimation cannot perform well in real-world challenging cases. We argue that it is particularly necessary for image deblurring to model degradation representations since blurry patterns typically show much larger variations than noisy patterns or high-frequency textures.In this paper, w…
Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark
Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education（教育部智能计算与信号处理重点实验室（安徽大学））；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）
Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
Recent years have witnessed the unprecedented success of deep convolutional neural networks (CNNs) in single image super-resolution (SISR). However, existing CNN-based SISR methods mostly assume that a low-resolution (LR) image is bicubicly downsampled from a high-resolution (HR) image, thus inevitably giving rise to poor performance when the true degradation does not follow this assumption. Moreover, they lack scalability in learning a single model to non-blindly deal with multiple degradations. To address these issues, we propose a general framework with dimensionality stretching strategy th…
Hyperspectral image (HSI) denoising is a crucial preprocessing procedure for the subsequent HSI applications. Unfortunately, though witnessing the development of deep learning in HSI denoising area, existing convolution-based methods face the trade-off between computational efficiency and capability to model non-local characteristics of HSI. In this paper, we propose a Spatial-Spectral Transformer (SST) to alleviate this problem. To fully explore intrinsic similarity characteristics in both spatial dimension and spectral dimension, we conduct non-local spatial self-attention and global spectra…
Fusing a low-resolution hyperspectral image (HSI) and a high-resolution multispectral image (MSI) of the same scene leads to a super-resolution image (SRI), which is information rich spatially and spectrally. In this paper, we super-resolve the HSI using the graph Laplacian defined on the MSI. Unlike many existing works, we don't assume prior knowledge about the spatial degradation from SRI to HSI, nor a perfectly aligned HSI and MSI pair. Our algorithm progressively alternates between finding the blur kernel and fusing HSI with MSI, generating accurate estimations of the blur kernel and the S…
Deep learning-based hyperspectral image super-resolution (SR) methods have achieved great success recently. However, most existing models can not effectively explore spatial information and spectral information between bands simultaneously, obtaining relatively low performance. To address this issue, in this paper, we propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet). Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential informatio…
Infrared Image Deturbulence Restoration Using Degradation Parameter-Assisted Wide & Deep Learning
Beihang University（北京航空航天大学）；Image Processing Center（图像处理中心）；The State Key Laboratory of Virtual Reality Technology and Systems（虚拟现实技术与系统国家重点实验室）；Beijing Advanced Innovation Center for Biomedical Engineering（北京生物医学工程高精尖创新中心）
Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院、教育部人工智能重点实验室）
John Hopcroft Center for Computer Science, Shanghai Jiao Tong University（上海交通大学约翰·霍普克罗夫特计算机科学中心）；Institute of Image Communication and Network Engineering,Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；Department of Electrical and Computer Engineering, University of Alabama（阿拉巴马大学电气与计算机工程系）；State Key Lab of Advanced Optical Communication Systems and Networks, Shanghai Jiao Tong University（上海交通大学先进光通信系统与网络国家重点实验室）
Current learning-based single image super-resolution (SISR) algorithms underperform on real data due to the deviation in the assumed degrada-tion process from that in the real-world scenario. Conventional degradation processes consider applying blur, noise, and downsampling (typicallybicubic downsampling) on high-resolution (HR) images to synthesize low-resolution (LR) counterparts. However, few works on degradation modelling have taken the physical aspects of the optical imaging system intoconsideration. In this paper, we analyze the imaging system optically andexploit the characteristics of …
ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Computer Science and Engineering, Macau University of Science and Technology（澳门科学技术大学计算机科学与工程学院）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Blind super-resolution (SR) methods aim to generate a high quality high resolution image from a low resolution image containing unknown degradations. However, natural images contain various types and amounts of blur: some may be due to the inherent degradation characteristics of the camera, but some may even be intentional, for aesthetic purposes (e.g. Bokeh effect). In the case of the latter, it becomes highly difficult for SR methods to disentangle the blur to remove, and that to leave as is. In this paper, we propose a novel blind SR framework based on kernel-oriented adaptive local adjustm…
UniUIR: Considering Underwater Image Restoration as An All-in-One Learner
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心，计算机学院，武汉大学）；School of Information Engineering, Guangdong University of Technology（信息工程学院，广东工业大学）；Horizon Robotics
Degradation-Aware Adaptive Context Gating for Unified Image Restoration
School of Computing and Artificial Intelligence, Southwest Jiaotong University（计算机与人工智能学院，西南交通大学）；Institute for Infocomm Research, Agency for Science, Technology and Research（信息通信研究院，科技研究局）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（计算机与人工智能学院，江西财经大学）；School of Information Management, Jiangxi University of Finance and Economics（信息管理学院，江西财经大学）；Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education（可持续智慧城市交通工程研究中心，教育部）；National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University（综合交通运输大数据应用技术国家工程实验室，西南交通大学）
