本文提出频域 HDR 融合网络 FHDRNet，利用 DWT 分离高低频，并通过注意力合并与频率引导上采样模块消除运动鬼影、保留细节，同时构建 RAW 数据集，实验达到最先进性能。
摘要：
从多张低动态范围（LDR）图像进行高动态范围（HDR）成像，长期受到场景和物体运动所导致的鬼影伪影困扰。现有方法，例如基于光流和基于端到端深度学习的方案，在细节恢复或鬼影伪影去除方面都容易出错。大量实证证据表明，由大幅前景运动引起的鬼影伪影主要是低频信号，而细节主要是高频信号。在本工作中，我们提出了一种新颖的频率引导端到端深度神经网络（FHDRNet），在频域中执行 HDR 融合，并使用离散小波变换（DWT）将输入分解为不同频带。低频信号用于避免特定鬼影伪影，而高频信号用于保留细节。我们以 U-Net 为主干，提出两个新颖模块：合并模块和频率引导上采样模块。合并模块将注意力机制应用于低频分量，以处理由大幅前景运动造成的鬼影。频率引导上采样模块从多个包含丰富细节的特定频率分量重建细节。此外，我们创建了一个新的 RAW 数据集，用于在 RAW 域中训练和评估多帧 HDR 成像算法。在公开数据集和我们的 RAW 数据集上进行了大量实验，结果表明所提出的 FHDRNet 达到了最先进性能。
英文摘要：
High dynamic range (HDR) imaging from multiple low dynamic range (LDR) images has been suffering from ghosting artifacts caused by scene and objects motion. Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. Comprehensive empirical evidence shows that ghosting artifacts caused by large foreground motion are mainly low-frequency signals and the details are mainly high-frequency signals. In this work, we propose a novel frequency-guided end-to-end deep neural network (FHDRNet) to conduct HDR fusion in the frequency domain, and Discrete Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details. Using a U-Net as the backbone, we propose two novel modules: merging module and frequency-guided upsampling module. The merging module applies the attention mechanism to the low-frequency components to deal with the ghost caused by large foreground motion. The frequency-guided upsampling module reconstructs details from multiple frequency-specific components with rich details. In addition, a new RAW dataset is created for training and evaluating multi-frame HDR imaging algorithms in the RAW domain. Extensive experiments are conducted on public datasets and our RAW dataset, showing that the proposed FHDRNet achieves state-of-the-art performance.
Nanchang University（南昌大学）；Paul C. Lauterbur Research Center for Biomedical Imaging, SIAT, CAS（中国科学院深圳先进技术研究院保罗·C·劳特伯生物医学成像研究中心）
The integration of compressed sensing and parallel imaging (CS-PI) provides a robust mechanism for accelerating MRI acquisitions. However, most such strategies require the explicit formation of either coil sensitivity profiles or a cross-coil correlation operator, and as a result reconstruction corresponds to solving a challenging bilinear optimization problem. In this work, we present an unsupervised deep learning framework for calibration-free parallel MRI, coined universal generative modeling for parallel imaging (UGM-PI). More precisely, we make use of the merits of both wavelet transform …
Optimal Hyperspectral Undersampling Strategy for Satellite Imaging
Bauman Moscow State Technical University（巴乌曼莫斯科国立技术大学）；VeinCV LLC（VeinCV公司）；Moscow Technical University of Communications and Informatics（莫斯科通信与信息学技术大学）；GIS（2GIS公司）；Artificial Intelligence Research Institute（人工智能研究 institutes）
Single-pixel imaging is an indirect imaging technique which utilizes simplified optical hardware and advanced computational methods. It offers novel solutions for hyper-spectral imaging, polarimetric imaging, three-dimensional imaging, holographic imaging, optical encryption and imaging through scattering media. The main limitations for its use come from relatively high measurement and reconstruction times. In this paper we propose to reduce the required signal acquisition time by using a novel sampling scheme based on a random selection of Morlet wavelets convolved with white noise. While suc…
Inspired by the extensive application of terahertz imaging technologies in the field of aerospace, we exploit a terahertz frequency modulated continuous wave imaging method with continuous wavelet transform algorithm to detect a multilayer heat shield made of special materials. This method uses the frequency modulation continuous wave system to catch the reflected terahertz signal and then processing the image data by the continuous wavelet transform with different basis functions. By calculating the sizes of the defects area in the final images and then comparing the results with real samples…
Recently, ghost imaging has been attracting attentions because its mechanism would lead to many applications inaccessible to conventional imaging methods. However, it is challenging for high contrast and high resolution imaging, due to its low signal-to-noise ratio (SNR) and the demand of high sampling rate in detection. To circumvent these challenges, we here propose a ghost imaging scheme that exploits Haar wavelets as illuminating patterns with a bi-frequency light projecting system and frequency-selecting single-pixel detectors. This method provides a theoretically 100% image contrast and …
Guizhou University（贵州大学）；Harbin Medical University Cancer Hospital（哈尔滨医科大学附属肿瘤医院）；Beijing Jiaotong University（北京交通大学）；Shanghai Jiao Tong University（上海交通大学）；University Lyon, INSA Lyon, CNRS, Inserm, IRP Metislab CREATIS UMR5220, U1206（里昂大学、里昂国立应用科学学院、法国国家科学研究中心、法国国家健康与医学研究院、IRP Metislab CREATIS UMR5220、U1206）
In vivo diffusion tensor imaging (DTI) is a promising technique to investigate noninvasively the fiber structures of the in vivo human heart. However, signal loss due to motions remains a persistent problem in in vivo cardiac DTI. We propose a novel motion-compensation method for investigating in vivo myocardium structures in DTI with free-breathing acquisitions. The method is based on an invertible Wavelet Scattering achieved by means of Convolutional Neural Network (WSCNN). It consists of first extracting translation-invariant wavelet scattering features from DW images acquired at different …
Magnetic Particle Imaging (MPI) is a preclinical imaging technique capable of visualizing the spatio-temporal distribution of magnetic nanoparticles. The image reconstruction of this fast and dynamic process relies on efficiently solving an ill-posed inverse problem. Current approaches to reconstruct the tracer concentration from its measurements are either adapted to image characteristics of MPI but suffer from higher computational complexity and slower convergence or are fast but lack in the image quality of the reconstructed images. In this work we propose a novel MPI reconstruction method …
This paper introduces a novel framework for single-pixel imaging via compressive sensing (CS) in shift-invariant (SI) spaces by exploiting the sparsity property of a wavelet representation. We reinterpret the acquisition procedure of a single-pixel camera as filtering of the observed signal with continuous-domain functions that lie in an SI subspace spanned by the integer shifts of the box function. The signal is modeled by an arbitrary SI generator whose special case is the box function, which, as we show in the paper, is conventionally used in single-pixel imaging. We propose to use separabl…
Purpose: Task-based assessment of image quality in undersampled magnetic resonance imaging provides a way of evaluating the impact of regularization on task performance. In this work, we evaluated the effect of total variation (TV) and wavelet regularization on human detection of signals with a varying background and validated a model observer in predicting human performance. Approach: Human observer studies used two-alternative forced choice (2-AFC) trials with a small signal known exactly task but with varying backgrounds for fluid-attenuated inversion recovery images reconstructed from unde…
Wavelet decompositions of integral operators have proven their efficiency in reducing computing times for many problems, ranging from the simulation of waves or fluids to the resolution of inverse problems in imaging. Unfortunately, computing the decomposition is itself a hard problem which is oftentimes out of reach for large scale problems. The objective of this work is to design fast decomposition algorithms based on another representation called product-convolution expansion. This decomposition can be evaluated efficiently assuming that a few impulse responses of the operator are available…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Korea Aerospace Research Institute (KARI)（韩国航空宇宙研究院）
Multi-spectral satellite imaging sensors acquire various spectral band images such as red (R), green (G), blue (B), near-infrared (N), etc. Thanks to the unique spectroscopic property of each spectral band with respective to the objects on the ground, multi-spectral satellite imagery can be used for various geological survey applications. Unfortunately, image artifacts from imaging sensor noises often affect the quality of scenes and have negative impacts on the applications of satellite imagery. Recently, deep learning approaches have been extensively explored for the removal of noises in sat…
3D Wavelet Latent Diffusion Model for Whole-Body MR-to-CT Modality Translation
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；School of Information Science and Technology, ShanghaiTech University（上海交通大学信息科学与技术学院）；Shanghai United Imaging Healthcare Co., Ltd.（上海联影医疗科技有限公司）；Department of Radiology and Biomedical Imaging, Yale University（耶鲁大学放射科与生物医学成像系）；Department of Applied Mathematics & Statistics and Department of Computer Science, Stony Brook University（石溪大学应用数学与统计系和计算机科学系）
Politecnico di Milano（米兰理工大学）；IMT Atlantique（大西洋矿业电信学院）
Graph Signal Processing has become a very useful framework for signal operations and representations defined on irregular domains. Exploiting transformations that are defined on graph models can be highly beneficial when the graph encodes relationships between signals. In this work, we present the benefits of using Spectral Graph Wavelet Transform (SGWT) as a feature extractor for machine learning on brain graphs. First, we consider a synthetic regression problem in which the smooth graph signals are generated as input with additive noise, and the target is derived from the input without noise…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Renmin University of China（中国人民大学）；Shenzhen University（深圳大学）
Magnetic resonance imaging plays an important role in computer-aided diagnosis and brain exploration. However, limited by hardware, scanning time and cost, it's challenging to acquire high-resolution (HR) magnetic resonance (MR) image clinically. In this paper, fine perceptive generative adversarial networks (FP-GANs) is proposed to produce HR MR images from low-resolution counterparts. It can cope with the detail insensitive problem of the existing super-resolution model in a divide-and-conquer manner. Specifically, FP-GANs firstly divides an MR image into low-frequency global approximation a…
Cryo-SWAN: the Multi-Scale Wavelet-decomposition-inspired Autoencoder Network for molecular density representation of molecular volumes
Center for Advanced Systems Understanding (CASUS)（先进系统理解中心（CASUS））；Helmholtz-Zentrum Dresden-Rossendorf e. V. (HZDR)（德累斯顿-罗斯托克亥姆霍尔茨研究中心（HZDR））；In situ Structural Biology（原位结构生物学）；Department of Physics（物理系）；Institute of Medical Physics and Biophysics（医学物理与生物物理研究所）；Institute of Computer Science（计算机科学研究所）；Cluster of Excellence Physics of Life（生命物理卓越中心）
Harvard-MIT Health Sciences and Technology, Massachusetts Institute of Technology（麻省理工学院哈佛-麻省理工健康科学与技术系）；Affective Computing group, MIT Media Lab, Massachusetts Institute of Technology（麻省理工学院媒体实验室情感计算研究组）；Center for Pain and the Brain, Harvard Medical School（哈佛医学院疼痛与大脑研究中心）；Department of Anesthesiology, Boston Children’s Hospital（波士顿儿童医院麻醉科）
Currently self-report pain ratings are the gold standard in clinical pain assessment. However, the development of objective automatic measures of pain could substantially aid pain diagnosis and therapy. Recent neuroimaging studies have shown the potential of functional near-infrared spectroscopy (fNIRS) for pain detection. This is a brain-imaging technique that provides non-invasive, long-term measurements of cortical hemoglobin concentration changes. In this study, we focused on fNIRS signals acquired exclusively from the prefrontal cortex, which can be accessed unobtrusively, and derived an …
