Universal Face Restoration With Memorized Modulation
National Laboratory of Pattern Recognition, CASIA（中国科学院自动化研究所模式识别国家重点实验室）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Blind face restoration (BFR) is a challenging problem because of the uncertainty of the degradation patterns. This paper proposes a Restoration with Memorized Modulation (RMM) framework for universal BFR in diverse degraded scenes and heterogeneous domains. We apply random noise as well as unsupervised wavelet memory to adaptively modulate the face-enhancement generator, considering attentional denormalization in both layer and instance levels. Specifically, in the training stage, the low-level spatial feature embedding, the wavelet memory embedding obtained by wavelet transformation of the high-resolution image, as well as the disentangled high-level noise embeddings are integrated, with the guidance of attentional maps generated from layer normalization, instance normalization, and the original feature map. These three embeddings are respectively associated with the spatial content, high-frequency texture details, and a learnable universal prior against other blind image degradation patterns. We store the spatial feature of the low-resolution image and the corresponding wavelet style code as key and value in the memory unit, respectively. In the test stage, the wavelet memory value whose corresponding spatial key is the most matching with that of the inferred image is retrieved to modulate the generator. Moreover, the universal prior learned from the random noise has been memorized by training the modulation network. Experimental results show the superiority of the proposed method compared with the state-of-the-art methods, and a good generalization in the wild.
M2IR: Proactive All-in-One Image Restoration via Mamba-style Modulation and Mixture-of-Experts
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Computer Science and Technology, Anhui University of Technology（安徽工业大学计算机科学与技术学院）；Shenzhen Ubiquitous Data Enabling Key Laboratory, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院深圳 ubiquitous 数据赋能重点实验室）；College of Artificial Intelligence, Taiyuan University of Technology（太原科技大学人工智能学院）
Modulating image restoration level aims to generate a restored image by altering a factor that represents the restoration strength. Previous works mainly focused on optimizing the mean squared reconstruction error, which brings high reconstruction accuracy but lacks finer texture details. This paper presents a Controllable Unet Generative Adversarial Network (CUGAN) to generate high-frequency textures in the modulation tasks. CUGAN consists of two modules -- base networks and condition networks. The base networks comprise a generator and a discriminator. In the generator, we realize the intera…
ClusIR: Towards Cluster-Guided All-in-One Image Restoration
Zhongnan University of Economics and Law（中南财经政法大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Yunnan University（云南大学）；Nanyang Technological University（南洋理工大学）；Wuhan University（武汉大学）
DocPure: Prompt-Free Unified Document Restoration via Degradation-Aware Structure-Guided Wavelet Modulation
College of Computer Science and Artificial Intelligence, Wenzhou University（温州大学计算机与人工智能学院）；AI Analytics Team, Nasdaq（纳斯达克人工智能分析团队）；vivo Mobile Communication Co., Ltd（维沃移动通信有限公司）；College of Engineering and Computer Science, The Australian National University（澳大利亚国立大学工程与计算机科学学院）
AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation
Technical University of Munich（慕尼黑工业大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Mohammed Bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；University of Central Florida（中佛罗里达大学）；Linköping University（林雪平大学）
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；The Chinese University of Hong Kong（香港中文大学）
In image restoration tasks, like denoising and super resolution, continual modulation of restoration levels is of great importance for real-world applications, but has failed most of existing deep learning based image restoration methods. Learning from discrete and fixed restoration levels, deep models cannot be easily generalized to data of continuous and unseen levels. This topic is rarely touched in literature, due to the difficulty of modulating well-trained models with certain hyper-parameters. We make a step forward by proposing a unified CNN framework that consists of few additional par…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）
Interactive image restoration aims to generate restored images by adjusting a controlling coefficient which determines the restoration level. Previous works are restricted in modulating image with a single coefficient. However, real images always contain multiple types of degradation, which cannot be well determined by one coefficient. To make a step forward, this paper presents a new problem setup, called multi-dimension (MD) modulation, which aims at modulating output effects across multiple degradation types and levels. Compared with the previous single-dimension (SD) modulation, the MD is …
UHDRes: Ultra-High-Definition Image Restoration via Dual-Domain Decoupled Spectral Modulation
College of Computer Science and Artificial Intelligence, Wenzhou University（计算机科学与人工智能学院，温州大学）；State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；College of Engineering and Computer Science, Australian National University（工程与计算机科学学院，澳大利亚国立大学）
SNNSIR: A Simple Spiking Neural Network for Stereo Image Restoration
School of Big Data and Software Engineering, Chongqing University（大数据与软件工程学院，重庆大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Microelectronics and Communication Engineering, Chongqing University（微电子与通信工程学院，重庆大学）；School of Electrical and Information Engineering, Tianjin University（电气与信息工程学院，天津大学）
Peking University Shenzhen Graduate School（北京大学深圳研究生院）；Peng Cheng Laboratory（鹏城实验室）；Tencent PCG（腾讯平台与内容事业群）；Shanghai AI Laboratory（上海人工智能实验室）
Interactive image restoration aims to restore images by adjusting several controlling coefficients, which determine the restoration strength. Existing methods are restricted in learning the controllable functions under the supervision of known degradation types and levels. They usually suffer from a severe performance drop when the real degradation is different from their assumptions. Such a limitation is due to the complexity of real-world degradations, which can not provide explicit supervision to the interactive modulation during training. However, how to realize the interactive modulation …
Intra and Inter Parser-Prompted Transformers for Effective Image Restoration
Sun Yat-sen University（中山大学）；Centre for Advances in Reliability and Safety（可靠性与安全先进中心）；The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Science and Technology（南京理工大学）；Dalian University of Technology（大连理工大学）
GTFMN: Guided Texture and Feature Modulation Network for Low-Light Image Enhancement and Super-Resolution
1 Graduate School of Engineering, Tohoku University, Sendai, Japan 2 Dept. of Radiology \& Biomedical Imaging, Yale University, New Haven, CT, USA 3 Dept. of Computer Science；Information Engineering, National Taiwan University, Taipei, Taiwan 4 Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan 5 Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan
Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Institute of Super Robotics(Huangpu)（超级机器人（黄埔）研究院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration
College of Computer Science, Nankai University（南开大学计算机学院）；Nankai International Advanced Research Institute（南开大学国际高等研究院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
Unifying Heterogeneous Degradations: Uncertainty-Aware Diffusion Bridge Model for All-in-One Image Restoration
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen, Guangdong 518107, China（中山大学智能系统工程学院深圳校区）；Department of Frontier Research, Peng Cheng Laboratory, Shenzhen, Guangdong 518055, China（鹏城实验室前沿研究部）；Guangdong Provincial Key Laboratory of Fire Science and Technology, Guangzhou 510006, China（广东省消防科学与技术重点实验室）
UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）
CoRE-UIR: Prior-guided common and residual experts for efficient all-in-one remote sensing image restoration
School of Geodesy and Geomatics, Wuhan University（武汉大学测绘学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation
Institute of Data Science, National Cheng Kung University（成功大学数据科学研究所）；Graduate Institute of Communication Engineering, National Taiwan University（台湾大学通讯工程学系）；NVIDIA（英伟达公司）
Efficient RAW Image Deblurring with Adaptive Frequency Modulation
School of Mathematics, Tianjin University（天津大学数学学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Macau（澳门大学）；Shanghai AI Laboratory（上海人工智能实验室）
Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes. On the premise that Point Spread Function (PSF…
URWKV: Unified RWKV Model with Multi-state Perspective for Low-light Image Restoration
Fujian Key Laboratory of Network Computing and Intelligent Information Processing, College of Computer and Data Science, Fuzhou University（福建网络计算与智能信息处理重点实验室，计算机与数据科学学院，福州大学）；Engineering Research Center of Big Data Intelligence, Ministry of Education（大数据智能工程研究中心，教育部）
School of Robotics, Hunan University（湖南大学机器人学院）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Technical University Munich（慕尼黑工业大学）
Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification
Fudan University（复旦大学）；Digital Medical Research Center（数字医学研究中心）；School of Basic Medical Science（基础医学院）；Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海医学影像计算与计算机辅助干预重点实验室）；Academy for Engineering and Technology（工程与技术学院）
CFRNet: Cycle-Consistent Fixed-Point Training for Real-Time Blind Face Restoration on Consumer Embedded NPUs
University of Florida（佛罗里达大学）；University of Southampton（南安普顿大学）；Chongqing University（重庆大学）；Qingdao University（青岛大学）；Intel Asia-Pacific Research & Development Ltd（英特尔亚太研发有限公司）
Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
CDSA-Net:Collaborative Decoupling of Vascular Structure and Background for High-Fidelity Coronary Digital Subtraction Angiography
School of Artificial Intelligence and Digital Economy Industry, Guangzhou Institute of Science and Technology（人工智能与数字经济学院，广州科学技术研究院）；Department of Cardiology, Second Affiliated Hospital of Nanchang University（南昌大学第二附属医院心内科）；School of Artificial Intelligence, Guangzhou Institute of Science and Technology（人工智能学院，广州科学技术研究院）
We propose an efficient neural network for RAW image denoising. Although neural network-based denoising has been extensively studied for image restoration, little attention has been given to efficient denoising for compute limited and power sensitive devices, such as smartphones and smartwatches. In this paper, we present a novel architecture and a suite of training techniques for high quality denoising in mobile devices. Our work is distinguished by three main contributions. (1) Feature-Align layer that modulates the activations of an encoder-decoder architecture with the input noisy images. …
Cross-modal Diffusion Modelling for Super-resolved Spatial Transcriptomics
Department of Clinical Neurosciences, University of Cambridge, UK（剑桥大学临床神经科学系）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（剑桥大学应用数学与理论物理系）；School of Science and Engineering, University of Dundee, UK（邓迪大学科学与工程学院）；School of Medicine, University of Dundee, UK（邓迪大学医学院）；Zhejiang Lab, China（浙江实验室）
