Image demosaicing is problem of interpolating full- resolution color images from raw sensor (color filter array) data. During last decade, deep neural networks have been widely used in image restoration, and in particular, in demosaicing, attaining significant performance improvement. In recent years, vision transformers have been designed and successfully used in various computer vision applications. One of the recent methods of image restoration based on a Swin Transformer (ST), SwinIR, demonstrates state-of-the-art performance with a smaller number of parameters than neural network-based methods. Inspired by the success of SwinIR, we propose in this paper a novel Swin Transformer-based network for image demosaicing, called RSTCANet. To extract image features, RSTCANet stacks several residual Swin Transformer Channel Attention blocks (RSTCAB), introducing the channel attention for each two successive ST blocks. Extensive experiments demonstrate that RSTCANet out- performs state-of-the-art image demosaicing methods, and has a smaller number of parameters.
In digital photography, two image restoration tasks have been studied extensively and resolved independently: demosaicing and super-resolution. Both these tasks are related to resolution limitations of the camera. Performing super-resolution on a demosaiced images simply exacerbates the artifacts introduced by demosaicing. In this paper, we show that such accumulation of errors can be easily averted by jointly performing demosaicing and super-resolution. To this end, we propose a deep residual network for learning an end-to-end mapping between Bayer images and high-resolution images. By traini…
West Virginia University（西弗吉尼亚大学）；Case Western Reserve University（凯斯西储大学）
Image demosaicing and super-resolution are two important tasks in color imaging pipeline. So far they have been mostly independently studied in the open literature of deep learning; little is known about the potential benefit of formulating a joint demosaicing and super-resolution (JDSR) problem. In this paper, we propose an end-to-end optimization solution to the JDSR problem and demonstrate its practical significance in computational imaging. Our technical contributions are mainly two-fold. On network design, we have developed a Residual-Dense Squeeze-and-Excitation Networks (RDSEN) supporte…
Image restoration is a long-standing low-level vision problem that aims to restore high-quality images from low-quality images (e.g., downscaled, noisy and compressed images). While state-of-the-art image restoration methods are based on convolutional neural networks, few attempts have been made with Transformers which show impressive performance on high-level vision tasks. In this paper, we propose a strong baseline model SwinIR for image restoration based on the Swin Transformer. SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image recons…
A framework of demosaicing and superresolution for color filter array (CFA) via residual image reconstruction and sparse representation is presented.Given the intermediate image produced by certain demosaicing and interpolation technique, a residual image between the final reconstruction image and the intermediate image is reconstructed using sparse representation.The final reconstruction image has richer edges and details than that of the intermediate image. Specifically, a generic dictionary is learned from a large set of composite training data composed of intermediate data and residual dat…
Spatial-Frequency Gated Swin Transformer for Cross-Sensor Remote Sensing Super-Resolution
Space Applications Centre, ISRO, Ahmedabad, India（印度航天研究组织阿赫迈德亚布德研究中心）；Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, India（印度理工学院孟买资源工程研究学院）；New L J Institute of Engineering and Technology, Ahmedabad, India（阿赫迈德亚布德新LJ工程与技术学院）；Pandit Deendayal Energy University, Gandhinagar, India（潘迪特·德恩达尔能源大学）；GLS University, Ahmedabad, India（阿赫迈德亚布德GLS大学）
In this paper, we propose a residual non-local attention network for high-quality image restoration. Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features. To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts. Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention b…
The quality of images captured by smartphones is an important specification since smartphones are becoming ubiquitous as primary capturing devices. The traditional image signal processing (ISP) pipeline in a smartphone camera consists of several image processing steps performed sequentially to reconstruct a high quality sRGB image from the raw sensor data. These steps consist of demosaicing, denoising, white balancing, gamma correction, colour enhancement, etc. Since each of them are performed sequentially using hand-crafted algorithms, the residual error from each processing module accumulate…
Couger Inc（库格公司）；Sethu Institute of Technology（塞图理工学院）；Thiagarajar college of Engineering（蒂亚加拉贾工程学院）
A moire pattern in the images is resulting from high frequency patterns captured by the image sensor (colour filter array) that appear after demosaicing. These Moire patterns would appear in natural images of scenes with high frequency content. The Moire pattern can also vary intensely due to a minimal change in the camera direction/positioning. Thus the Moire pattern depreciates the quality of photographs. An important issue in demoireing pattern is that the Moireing patterns have dynamic structure with varying colors and forms. These challenges makes the demoireing more difficult than many o…
GRFormer: Grouped Residual Self-Attention for Lightweight Single Image Super-Resolution
Tsinghua University（清华大学）；Swinburne University of Technology（斯威本科技大学）；National University of Singapore（新加坡国立大学）；University of International Business and Economics（对外经济贸易大学）
University of Cambridge（剑桥大学）；Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）
Super-resolution plays an essential role in medical imaging because it provides an alternative way to achieve high spatial resolutions and image quality with no extra acquisition costs. In the past few decades, the rapid development of deep neural networks has promoted super-resolution performance with novel network architectures, loss functions and evaluation metrics. Specifically, vision transformers dominate a broad range of computer vision tasks, but challenges still exist when applying them to low-level medical image processing tasks. This paper proposes an efficient vision transformer wi…
Taiyuan University of Science and Technology（太原科技大学）；Jiangnan University（江南大学）；Nanyang Normal University（南阳师范学院）
The existing deep learning fusion methods mainly concentrate on the convolutional neural networks, and few attempts are made with transformer. Meanwhile, the convolutional operation is a content-independent interaction between the image and convolution kernel, which may lose some important contexts and further limit fusion performance. Towards this end, we present a simple and strong fusion baseline for infrared and visible images, namely\textit{ Residual Swin Transformer Fusion Network}, termed as SwinFuse. Our SwinFuse includes three parts: the global feature extraction, fusion layer and fea…
Sun Yat-sen University（中山大学）；China Three Gorges University（三峡大学）；The First People’s Hospital of Yichang（宜昌市第一人民医院）
Decreasing projection views to lower X-ray radiation dose usually leads to severe streak artifacts. To improve image quality from sparse-view data, a Multi-domain Integrative Swin Transformer network (MIST-net) was developed in this article. First, MIST-net incorporated lavish domain features from data, residual-data, image, and residual-image using flexible network architectures, where residual-data and residual-image sub-network was considered as data consistency module to eliminate interpolation and reconstruction errors. Second, a trainable edge enhancement filter was incorporated to detec…
Robert Bosch Center of Data Science and AI, IIT Madras（印度理工学院马德拉斯分校罗伯特·博世数据科学与人工智能中心）；Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）
In this work, we present our proposed method to segment the pulmonary arteries from the CT scans using Swin UNETR and U-Net-based deep neural network architecture. Six models, three models based on Swin UNETR, and three models based on 3D U-net with residual units were ensemble using a weighted average to make the final segmentation masks. Our team achieved a multi-level dice score of 84.36 percent through this method. The code of our work is available on the following link: https://github.com/akansh12/parse2022. This work is part of the MICCAI PARSE 2022 challenge.
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）
This paper studies the challenging problem of recovering motion from blur, also known as joint deblurring and interpolation or blur temporal super-resolution. The challenges are twofold: 1) the current methods still leave considerable room for improvement in terms of visual quality even on the synthetic dataset, and 2) poor generalization to real-world data. To this end, we propose a blur interpolation transformer (BiT) to effectively unravel the underlying temporal correlation encoded in blur. Based on multi-scale residual Swin transformer blocks, we introduce dual-end temporal supervision an…
A conventional camera performs various signal processing steps sequentially to reconstruct an image from a raw Bayer image. When performing these processing in multiple stages the residual error from each stage accumulates in the image and degrades the quality of the final reconstructed image. In this paper, we present a fully convolutional neural network (CNN) to perform defect pixel correction, denoising, white balancing, exposure correction, demosaicing, color transform, and gamma encoding. To our knowledge, this is the first CNN trained end-to-end to perform the entire image signal process…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Microsoft（微软）
Mixture of Experts (MoE) is able to scale up vision transformers effectively. However, it requires prohibiting computation resources to train a large MoE transformer. In this paper, we propose Residual Mixture of Experts (RMoE), an efficient training pipeline for MoE vision transformers on downstream tasks, such as segmentation and detection. RMoE achieves comparable results with the upper-bound MoE training, while only introducing minor additional training cost than the lower-bound non-MoE training pipelines. The efficiency is supported by our key observation: the weights of an MoE transforme…
QUIET-SR: Quantum Image Enhancement Transformer for Single Image Super-Resolution
College of Computing \& Data Science, Nanyang Technological University (NTU), Singapore, 639798, Singapore；SVKM's Dwarkadas J. Sanghvi College of Engineering, Mumbai, India；eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE；Center for Quantum；Topological Systems (CQTS), NYUAD Research Institute, NYUAD, Abu Dhabi, UAE；IBM Quantum, IBM T.J. Watson Research Center, Yorktown Heights, 10598, USA；MIT-IBM Watson AI Lab, Cambridge MA, 02142, USA；The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Sønderborg, Denmark；Department of Software Science, Tallinn University of Technology, Tallinn, Estonia
Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
School of Informatics, Xiamen University（厦门大学信息学院）；School of Computer Science（计算机科学学院）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；SSE, The Chinese University of Hong Kong（香港中文大学（深圳）SSE）
Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）；Sun Yat-sen University（中山大学）；Beihang University（北京航空航天大学）；University of the Basque Country UPV/EHU（巴斯克大学（UPV/EHU））；TECNALIA（泰克纳利亚研究中心）；Shenzhen Second People’s Hospital（深圳市第二人民医院）；The First Affiliated Hospital of Shenzhen University Health Science Center（深圳大学医学部第一附属医院）
Magnetic resonance imaging (MRI) is an important non-invasive clinical tool that can produce high-resolution and reproducible images. However, a long scanning time is required for high-quality MR images, which leads to exhaustion and discomfort of patients, inducing more artefacts due to voluntary movements of the patients and involuntary physiological movements. To accelerate the scanning process, methods by k-space undersampling and deep learning based reconstruction have been popularised. This work introduced SwinMR, a novel Swin transformer based method for fast MRI reconstruction. The who…
Large-scale NLP models have been shown to significantly improve the performance on language tasks with no signs of saturation. They also demonstrate amazing few-shot capabilities like that of human beings. This paper aims to explore large-scale models in computer vision. We tackle three major issues in training and application of large vision models, including training instability, resolution gaps between pre-training and fine-tuning, and hunger on labelled data. Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) …
Res-MoCoDiff: Residual-guided diffusion models for motion artifact correction in brain MRI
Department of Radiation Oncology and Winship Cancer Institute, Emory University（放射肿瘤学系和Winship癌症研究所，埃默里大学）；Department of Radiology and Image Science and Winship Cancer Institute, Emory University（放射学系和影像科学以及Winship癌症研究所，埃默里大学）
A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery
Wayne State University（韦恩州立大学）；Henry Ford Health（亨利福特医疗集团）；Alexandria University（亚历山大大学）；Michigan State University（密歇根州立大学）；Oakland University（奥克兰大学）；Institute for AI and Data Science, Wayne State University（韦恩州立大学人工智能与数据科学学院）
