WaveFuse: A Unified Deep Framework for Image Fusion with Discrete Wavelet Transform
School of Basic Medical Science, Fudan University（复旦大学基础医学院）；Shanghai Key Laboratory of Medical Image Computing and Computer Assited Intervention（上海市医学图像计算与计算机辅助干预重点实验室）
Shaolei Liu, Manning Wang, Zhijian Song
备注：
accepted at The 28th International Conference on Neural Information Processing(ICONIP2021)
We propose an unsupervised image fusion architecture for multiple application scenarios based on the combination of multi-scale discrete wavelet transform through regional energy and deep learning. To our best knowledge, this is the first time the conventional image fusion method has been combined with deep learning. The useful information of feature maps can be utilized adequately through multi-scale discrete wavelet transform in our proposed method.Compared with other state-of-the-art fusion method, the proposed algorithm exhibits better fusion performance in both subjective and objective evaluation. Moreover, it's worth mentioning that comparable fusion performance trained in COCO dataset can be obtained by training with a much smaller dataset with only hundreds of images chosen randomly from COCO. Hence, the training time is shortened substantially, leading to the improvement of the model's performance both in practicality and training efficiency.
Among the existing fusion algorithms, the wavelet fusion method is the most frequently discussed one in recent publications because the wavelet approach preserves the spectral characteristics of the multispectral image better than other methods. The Brovey is also a popular fusion method used for its ability in preserving the spatial information of the PAN image. This study presents a new fusion approach that integrates the advantages of both the Brovey (which preserves a high degree of spatial information) and the wavelet (which preserves a high degree of spectral information) techniques to r…
Image fusion produces a single fused image from a set of input images. A new method for image fusion is proposed based on Weighted Average Merging Method (WAMM) in the NonSubsampled Contourlet Transform (NSCT) domain. A performance analysis on various statistical fusion rules are also analysed both in NSCT and Wavelet domain. Analysis has been made on medical images, remote sensing images and multi focus images. Experimental results shows that the proposed method, WAMM obtained better results in NSCT domain than the wavelet domain as it preserves more edges and keeps the visual quality intact …
SSPM's College of Engineering, Kankavali（SSPM工程学院）
The idea of combining multiple image modalities to provide a single, enhanced image is well established different fusion methods have been proposed in literature. This paper is based on image fusion using laplacian pyramid and wavelet transform method. Images of same size are used for experimentation. Images used for the experimentation are standard images and averaging filter is used of equal weights in original images to burl. Performance of image fusion technique is measured by mean square error, normalized absolute error and peak signal to noise ratio. From the performance analysis it has …
In an appropriate image fusion method, spatial information of the panchromatic image is injected into the multispectral images such that the spectral information is not distorted. The high-pass modulation method is a successful method in image fusion. However, the main drawback of this method is that this technique uses the boxcar filter to extract the high frequency information of the panchromatic image. Using the boxcar filter introduces the ringing effect into the fused image. To cope with this problem, we use the wavelet transform instead of boxcar filters. Then, the results of the propose…
This report presents the results of a multi-scale wavelet based scheme for single image de-hazing and underwater image enhancement. The scheme is fast and highly localized in addition to global enhancement of hazy images. A PDE-based formulation enables additional versatility as the iterative nature allows more flexibility for various types of images. Visual and objective results from experiments indicate that the proposed approach competes favourably or surpasses most of the state-of-the-art approaches.
RVS College of Engineering and Technology（RVS工程技术学院）
In recent years, many research achievements are made in the medical image fusion field. Fusion is basically extraction of best of inputs and conveying it to the output. Medical Image fusion means that several of various modality image information is comprehended together to form one image to express its information. The aim of image fusion is to integrate complementary and redundant information. In this paper, a multimodal image fusion algorithm based on the dual-tree complex wavelet transform (DT-CWT) and adaptive particle swarm optimization (APSO) is proposed. Fusion is achieved through the …
This paper presents the Discrete Wavelet based fusion techniques for combining perceptually important image features. SPIHT (Set Partitioning in Hierarchical Trees) algorithm is an efficient method for lossy and lossless coding of fused image. This paper presents some modifications on the SPIHT algorithm. It is based on the idea of insignificant correlation of wavelet coefficient among the medium and high frequency sub bands. In RE-MSPIHT algorithm, wavelet coefficients are scaled prior to SPIHT coding based on the sub band importance, with the goal of minimizing the MSE.
Multi-modal Medical Neurological Image Fusion using Wavelet Pooled Edge Preserving Autoencoder
Visvesvaraya National Institute of Technology Nagpur（维斯瓦瓦拉亚那格浦尔国家理工学院）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；All India Institute of Medical Sciences Nagpur（全印那格浦尔医学科学研究所）
We present a wavelet-based dual-stream network that addresses color cast and blurry details in underwater images. We handle these artifacts separately by decomposing an input image into multiple frequency bands using discrete wavelet transform, which generates the downsampled structure image and detail images. These sub-band images are used as input to our dual-stream network that incorporates two sub-networks: the multi-color space fusion network and the detail enhancement network. The multi-color space fusion network takes the decomposed structure image as input and estimates the color corre…
We propose a fusion algorithm for haze removal that combines color information from an RGB image and edge information extracted from its corresponding NIR image using Haar wavelets. The proposed algorithm is based on the key observation that NIR edge features are more prominent in the hazy regions of the image than the RGB edge features in those same regions. To combine the color and edge information, we introduce a haze-weight map which proportionately distributes the color and edge information during the fusion process. Because NIR images are, intrinsically, nearly haze-free, our work makes …
Taibah University（塔伊巴大学）；Donetsk National Technical University（顿涅茨克国立技术大学）
The paper presents a parallel implementation of existing image fusion methods on a graphical cluster. Parallel implementations of methods based on discrete wavelet transformation (Haars and Daubechies discrete wavelet transform) are developed. Experiments were performed on a cluster using GPU and CPU and performance gains were estimated for the use of the developed parallel implementations to process satellite images from satellite Landsat 7. The implementation on a graphic cluster provides performance improvement from 2 to 18 times. The quality of the considered methods was evaluated by ERGAS…
Image registration is the process of transforming different sets of data into one coordinate system and is required for various remote sensing applications like change detection, image fusion, and other related areas. The effect of increased relief displacement, requirement of more control points, and increased data volume are the challenges associated with the registration of high resolution image data. The objective of this research work is to study the most efficient techniques and to investigate the extent of improvement achievable by enhancing them with Wavelet transform. The SIFT feature…
SSN College of Engineering（SSN工程学院）；RMD Engineering College（RMD工程学院）
Image fusion plays a vital role in medical imaging. Image fusion aims to integrate complementary as well as redundant information from multiple modalities into a single fused image without distortion or loss of information. In this research work, discrete wavelet transform (DWT)and undecimated discrete wavelet transform (UDWT)-based fusion techniques using genetic algorithm (GA)foroptimalparameter(weight)estimationinthefusionprocessareimplemented and analyzed with multi-modality brain images. The lack of shift variance while performing image fusion using DWT is addressed using UDWT. The propos…
This paper describes a new method for mitigating the effects of atmospheric distortion on observed sequences that include large moving objects. In order to provide accurate detail from objects behind the distorting layer, we solve the space-variant distortion problem using recursive image fusion based on the Dual Tree Complex Wavelet Transform (DT-CWT). The moving objects are detected and tracked using the improved Gaussian mixture models (GMM) and Kalman filtering. New fusion rules are introduced which work on the magnitudes and angles of the DT-CWT coefficients independently to achieve a sha…
Center of Multidisciplinary Integrated Technologies for Field Monitoring（现场监测多学科综合技术中心）
Impervious surface is an important indicator for urban development monitoring. Accurate urban impervious surfaces mapping with VNREDSat-1 remains challenging due to their spectral diversity not captured by individual PAN image. In this artical, five multi-resolution image fusion techniques were compared for classification task of urban impervious surface. The result shows that for VNREDSat-1 dataset, UNB and Wavelet tranform methods are the best techniques reserving spatial and spectral information of original MS image, respectively. However, the UNB technique gives best results when it comes …
CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration
School of Physics（物理学院）；Optoelectronic Engineering, Foshan University, Foshan 528225, China（光学电子工程学院，佛山大学，佛山528225，中国）；Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China（粤港澳联合智能微纳光电子技术实验室，佛山528225，中国）；School of Mechanical Electronic（机械电子学院）；Information Engineering, China University of Mining and Technology, Beijing 100083, China（信息工程学院，中国矿业大学，北京100083，中国）
In this paper we present a comparative study on fusion of visual and thermal images using different wavelet transformations. Here, coefficients of discrete wavelet transforms from both visual and thermal images are computed separately and combined. Next, inverse discrete wavelet transformation is taken in order to obtain fused face image. Both Haar and Daubechies (db2) wavelet transforms have been used to compare recognition results. For experiments IRIS Thermal/Visual Face Database was used. Experimental results using Haar and Daubechies wavelets show that the performance of the approach pres…
ASUS Intelligent Cloud Services（华硕智能云服务）；Georgia Institute of Technology（佐治亚理工学院）；National Taiwan University（台湾大学）
Single image deraining is a crucial problem because rain severely degenerates the visibility of images and affects the performance of computer vision tasks like outdoor surveillance systems and intelligent vehicles. In this paper, we propose the new convolutional neural network (CNN) called the wavelet channel attention module with a fusion network. Wavelet transform and the inverse wavelet transform are substituted for down-sampling and up-sampling so feature maps from the wavelet transform and convolutions contain different frequencies and scales. Furthermore, feature maps are integrated by …
Scientific Research Group in Egypt (SRGE)（埃及科学研究组）；Benha University（本哈大学）；Damietta University（达米埃塔大学）；Cairo University（开罗大学）
Remote sensing image fusion is an effective way to use a large volume of data from multisensor images. Most earth satellites such as SPOT, Landsat 7, IKONOS and QuickBird provide both panchromatic (Pan) images at a higher spatial resolution and multispectral (MS) images at a lower spatial resolution and many remote sensing applications require both high spatial and high spectral resolutions, especially for GIS based applications. An effective image fusion technique can produce such remotely sensed images. Image fusion is the combination of two or more different images to form a new image by us…
In this paper fusion of visual and thermal images in wavelet transformed domain has been presented. Here, Daubechies wavelet transform, called as D2, coefficients from visual and corresponding coefficients computed in the same manner from thermal images are combined to get fused coefficients. After decomposition up to fifth level (Level 5) fusion of coefficients is done. Inverse Daubechies wavelet transform of those coefficients gives us fused face images. The main advantage of using wavelet transform is that it is well-suited to manage different image resolution and allows the image decomposi…
In remote sensing, image fusion technique is a useful tool used to fuse high spatial resolution panchromatic images (PAN) with lower spatial resolution multispectral images (MS) to create a high spatial resolution multispectral of image fusion (F) while preserving the spectral information in the multispectral image (MS).There are many PAN sharpening techniques or Pixel-Based image fusion techniques that have been developed to try to enhance the spatial resolution and the spectral property preservation of the MS. This paper attempts to undertake the study of image fusion, by using two types of …
University of Wollongong（伍伦贡大学）；Microsoft Research（微软研究院）；Nanjing Forestry University（南京林业大学）
Detecting camouflaged moving foreground objects has been known to be difficult due to the similarity between the foreground objects and the background. Conventional methods cannot distinguish the foreground from background due to the small differences between them and thus suffer from under-detection of the camouflaged foreground objects. In this paper, we present a fusion framework to address this problem in the wavelet domain. We first show that the small differences in the image domain can be highlighted in certain wavelet bands. Then the likelihood of each wavelet coefficient being foregro…
WaveDH: Wavelet Sub-bands Guided ConvNet for Efficient Image Dehazing
Artificial Intelligence Graduate School, Gwangju Institute of Science and Technology (GIST)（光州科学技术院人工智能研究生院）；School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST)（光州科学技术院电气与计算机工程学院）
Central University of Finance and Economics（中央财经大学）
The GAN-based infrared and visible image fusion methods have gained ever-increasing attention due to its effectiveness and superiority. However, the existing methods adopt the global pixel distribution of source images as the basis for discrimination, which fails to focus on the key modality information. Moreover, the dual-discriminator based methods suffer from the confrontation between the discriminators. To this end, we propose an attention-guided and wavelet-constrained GAN for infrared and visible image fusion (AWFGAN). In this method, two unique discrimination strategies are designed to …
Task-Generalized Adaptive Cross-Domain Learning for Multimodal Image Fusion
Key Laboratory of Opto-Electronic Information Science and Technology of Jiangxi Province, Nanchang Hangkong University（江西省光电信息科学与技术重点实验室，南昌航空大学）；ReLER, CCAI, Zhejiang University（ReLER、CCAI、浙江大学）；College of Engineering, Anhui Agricultural University（安徽农业大学工程学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）
This paper investigates the multiresolution level-1 and level-2 Quotient based Fusion of thermal and visual images. In the proposed system, the method-1 namely "Decompose then Quotient Fuse Level-1" and the method-2 namely "Decompose-Reconstruct then Quotient Fuse Level-2" both work on wavelet transformations of the visual and thermal face images. The wavelet transform is well-suited to manage different image resolution and allows the image decomposition in different kinds of coefficients, while preserving the image information without any loss. This approach is based on a definition of an ill…
VNRVJIET（瓦利瓦纳蒂·纳加尔朱纳·拉奥·维贾雅·贾瓦哈尔·工程技术学院）；GNITS（吉特工程学院）；JNTU College of Engineering（贾瓦哈拉尔·尼赫鲁科技大学工程学院）
Image fusion is to reduce uncertainty and minimize redundancy in the output while maximizing relevant information from two or more images of a scene into a single composite image that is more informative and is more suitable for visual perception or processing tasks like medical imaging, remote sensing, concealed weapon detection, weather forecasting, biometrics etc. Image fusion combines registered images to produce a high quality fused image with spatial and spectral information. The fused image with more information will improve the performance of image analysis algorithms used in different…
A Wavelet Guided Attention Module for Skin Cancer Classification with Gradient-based Feature Fusion
Jadavpur University（jadavpur大学）；International Institute of Information Technology（国际信息技术学院）；China University of Mining and Technology（中国矿业大学）；Saint Petersburg Electrotechnical University "LETI"（圣彼得堡电工技术大学“LETI”）
Wavelet-Fusion Diffusion Model for Multimodal Brain MRI Synthesis with Modality and Metadata Conditioning
Department of Psychology & Neuroscience, Auckland University of Technology（心理学与神经科学系，奥克兰技术大学）；Department of Psychiatry, University of Melbourne（精神病学系，墨尔本大学）
Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detection
Thrust of Artificial Intelligence, Hong Kong University of Science and Technology (Guangzhou)（人工智能 thrust，香港科技大学（广州））；Momoniai AI；Department of Mechanical Engineering, Massachusetts Institute of Technology（机械工程系，麻省理工学院）；School of Information Engineering, Yancheng Institute of Technology（信息工程学院，盐城科技学院）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）；Institute for Math & AI, Wuhan University（数学与人工智能研究所，武汉大学）；College of Science and Engineering and the Centre for AI and Data Science Innovation, James Cook University（科学与工程学院及人工智能与数据科学创新中心，詹姆斯库克大学）；School of Transportation, Southeast University（交通运输学院，东南大学）
Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion
Institute of Computational Imaging, Beijing Information Science and Technology University（北京信息科技大学计算成像研究所）；College of Computer Science (College of Software), Inner Mongolia University（内蒙古大学计算机学院（软件学院））；Research Center for Spatiotemporal Intelligence, Inner Mongolia University（内蒙古大学时空智能研究中心）
This thesis describes a study to perform change detection on Very High Resolution satellite images using image fusion based on 2D Discrete Wavelet Transform and Fuzzy C-Means clustering algorithm. Multiple other methods are also quantitatively and qualitatively compared in this study.
Wavelet-Assisted Multi-Frequency Attention Network for Pansharpening
University of Electronic Science and Technology of China（电子科技大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
An ideal fusion method preserves the Spectral information in fused image and adds spatial information to it with no spectral distortion. Recently wavelet kalman filter method is proposed which uses ARSIS concept to fuses MS and PAN images. This method is applied in a multiscale version, i.e. the variable index is scale instead of time. With the aim of fusion we present a more detailed study on this model and discuss about rationality of its assumptions such as first order markov model and Gaussian distribution of the posterior density. Finally, we propose a method using wavelet Kalman Particle…
This paper demonstrates two different fusion techniques at two different levels of a human face recognition process. The first one is called data fusion at lower level and the second one is the decision fusion towards the end of the recognition process. At first a data fusion is applied on visual and corresponding thermal images to generate fused image. Data fusion is implemented in the wavelet domain after decomposing the images through Daubechies wavelet coefficients (db2). During the data fusion maximum of approximate and other three details coefficients are merged together. After that Prin…
Wavelet-based Global-Local Interaction Network with Cross-Attention for Multi-View Diabetic Retinopathy Detection
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Shenzhen Key Laboratory of Visual Object Detection and Recognition（深圳市视觉目标检测与识别重点实验室）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）
WCCNet: Wavelet-context Cooperative Network for Efficient Multispectral Pedestrian Detection
the College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；the College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）
WaveRNet: Wavelet-Guided Frequency Learning for Multi-Source Domain-Generalized Retinal Vessel Segmentation
School of Software, Xinjiang University, Urumqi, Xinjiang, China（新疆大学软件学院）；The First Clinical Medical College, Nanjing University of Chinese Medicine, Nanjing, China（南京中医药大学第一临床医学院）；School of Computer Science, University of Nottingham, UK（诺丁汉大学计算机科学学院）
In this paper, a novel method for representation and recognition of the facial expressions in two-dimensional image sequences is presented. We apply a variation of two-dimensional heteroscedastic linear discriminant analysis (2DHLDA) algorithm, as an efficient dimensionality reduction technique, to Gabor representation of the input sequence. 2DHLDA is an extension of the two-dimensional linear discriminant analysis (2DLDA) approach and it removes the equal within-class covariance. By applying 2DHLDA in two directions, we eliminate the correlations between both image columns and image rows. The…
