Polar Fusion Technique Analysis for Evaluating the Performances of Image Fusion of Thermal and Visual Images for Human Face Recognition
Tripura University（特里普拉大学）；Jadavpur University（贾达普大学）；Université Pierre et Marie Curie, Paris-6（皮埃尔和玛丽·居里大学（巴黎第六大学））；Université Denis Diderot, Paris-7（丹尼斯·狄德罗大学（巴黎第七大学））
Mrinal Kanti Bhowmik, Debotosh Bhattacharjee, Dipak Kumar Basu, Mita Nasipuri
备注：
Proceedings of IEEE Workshop on Computational Intelligence in Biometrics and Identity Management (IEEE CIBIM 2011), Paris, France, April 11 - 15, 2011
This paper presents a comparative study of two different methods, which are based on fusion and polar transformation of visual and thermal images. Here, investigation is done to handle the challenges of face recognition, which include pose variations, changes in facial expression, partial occlusions, variations in illumination, rotation through different angles, change in scale etc. To overcome these obstacles we have implemented and thoroughly examined two different fusion techniques through rigorous experimentation. In the first method log-polar transformation is applied to the fused images obtained after fusion of visual and thermal images whereas in second method fusion is applied on log-polar transformed individual visual and thermal images. After this step, which is thus obtained in one form or another, Principal Component Analysis (PCA) is applied to reduce dimension of the fused images. Log-polar transformed images are capable of handling complicacies introduced by scaling and rotation. The main objective of employing fusion is to produce a fused image that provides more detailed and reliable information, which is capable to overcome the drawbacks present in the individual visual and thermal face images. Finally, those reduced fused images are classified using a multilayer perceptron neural network. The database used for the experiments conducted here is Object Tracking and Classification Beyond Visible Spectrum (OTCBVS) database benchmark thermal and visual face images. The second method has shown better performance, which is 95.71% (maximum) and on an average 93.81% as correct recognition rate.
The data fusion technology aims to aggregate the characteristics of different data and obtain products with multiple data advantages. To solves the problem of reduced resolution of PolSAR images due to system limitations, we propose a fully polarimetric synthetic aperture radar (PolSAR) images and single-polarization synthetic aperture radar SAR (SinSAR) images fusion network to generate high-resolution PolSAR (HR-PolSAR) images. To take advantage of the polarimetric information of the low-resolution PolSAR (LR-PolSAR) image and the spatial information of the high-resolution single-polarizatio…
A Luminance-Aware Multi-Scale Network for Polarization Image Fusion with a Multi-Scene Dataset
Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University（广东-香港-澳门智能微纳光电子技术联合实验室，物理与光电工程学院，佛山大学）；State Key Laboratory of Dynamic Measurement Technology, North University of China（动态测试技术国家重点实验室，中北大学）；School of Information Engineering and Automation, Kunming University of Science and Technology（信息工程与自动化学院，昆明理工大学）
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心（DLR）遥感技术研究所（IMF））；Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM)（慕尼黑工业大学（TUM）地球观测信号处理实验室（SiPEO））
Multi-modal data fusion has recently been shown promise in classification tasks in remote sensing. Optical data and radar data, two important yet intrinsically different data sources, are attracting more and more attention for potential data fusion. It is already widely known that, a machine learning based methodology often yields excellent performance. However, the methodology relies on a large training set, which is very expensive to achieve in remote sensing. The semi-supervised manifold alignment (SSMA), a multi-modal data fusion algorithm, has been designed to amplify the impact of an exi…
Multimodal Attention-Aware Fusion for Diagnosing Distal Myopathy: Evaluating Model Interpretability and Clinician Trust
Information Systems Group, Eindhoven University of Technology, The Netherlands（埃因霍温技术大学信息系统组）；Eindhoven Artificial Intelligence Systems Institute, The Netherlands（埃因霍温人工智能系统研究所）；Department of Computer Science and Technology, University of Cambridge, United Kingdom（剑桥大学计算机科学与技术系）；Department of Medicine - DIMED, Padua University Hospital, Italy（帕多瓦大学医院医学部-DIMED）；Department of Environmental Sciences, Informatics, and Statistics, Ca’ Foscari University of Venice, Italy（威尼斯卡弗里大学环境科学、信息学与统计学系）
A Polarization and Radiomics Feature Fusion Network for the Classification of Hepatocellular Carcinoma and Intrahepatic Cholangiocarcinoma
University College London（伦敦大学学院）；Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Fujian Medical University Cancer Hospital（福建医科大学肿瘤医院）；Fujian Cancer Hospital（福建省肿瘤医院）
Yeshwant Mahavidyala College (SRTMU)（耶什万特·马哈维迪亚拉学院（SRTMU））
Until now, of highest relevance for remote sensing data processing and analysis have been techniques for pixel level image fusion. So, This paper attempts to undertake the study of Feature-Level based image fusion. For this purpose, feature based fusion techniques, which are usually based on empirical or heuristic rules, are employed. Hence, in this paper we consider feature extraction (FE) for fusion. It aims at finding a transformation of the original space that would produce such new features, which preserve or improve as much as possible. This study introduces three different types of Imag…
Beyond Discrimination: Evaluating Probability Calibration in Radiomics-Image Fusion for Renal Cell Carcinoma Classification
Research Ireland Centre for Research Training in Machine Learning（爱尔兰研究机器学习研究中心）；ML-Labs（机器学习实验室）；School of Computer Science, University College Dublin（都柏林大学计算机学院）；School of Computer Science, Technological University Dublin（都柏林理工大学计算机学院）；School of Veterinary Medicine, University College Dublin（都柏林大学兽医学院）；School of Computing, Dublin City University（都柏林城市大学计算机学院）
Zhejiang University（浙江大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Semantic Segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on. However, traditional SS is primarily based on RGB images, which limits the reliability of SS in complex outdoor scenes, where RGB images lack necessary information dimensions to fully perceive unconstrained environments. As preliminary investigation, we examine SS in an unexpected obstacle detection scenario, which demonstrates the necessity of multimodal fusion. Thereby, in this work, we present EAFNet, an Efficient Attention-bridged Fus…
Yeshwant Mahavidyala College (SRTMU)（耶什万特·马哈维亚拉学院（SRTMU））
There are many image fusion methods that can be used to produce high-resolution mutlispectral images from a high-resolution panchromatic (PAN) image and low-resolution multispectral (MS) of remote sensed images. This paper attempts to undertake the study of image fusion techniques with different Statistical techniques for image fusion as Local Mean Matching (LMM), Local Mean and Variance Matching (LMVM), Regression variable substitution (RVS), Local Correlation Modeling (LCM) and they are compared with one another so as to choose the best technique, that can be applied on multi-resolution sate…
In remote sensing, image fusion technique is a useful tool used to fuse high spatial resolution panchromatic images (PAN) with lower spatial resolution multispectral images (MS) to create a high spatial resolution multispectral of image fusion (F) while preserving the spectral information in the multispectral image (MS).There are many PAN sharpening techniques or Pixel-Based image fusion techniques that have been developed to try to enhance the spatial resolution and the spectral property preservation of the MS. This paper attempts to undertake the study of image fusion, by using two types of …
Rutgers, The State University of New Jersey（罗格斯大学新泽西州立大学）；U.S. Army Research Laboratory（美国陆军研究实验室）；Booz Allen Hamilton（博思艾伦汉密尔顿公司）；Johns Hopkins University（约翰斯·霍普金斯大学）
The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face verification a highly challenging problem for human examiners as well as computer vision algorithms. Previous approaches utilize either a two-step procedure (visible feature estimation and visible image reconstruction) or an input-level fusion technique, where different Stokes images are concatenated and used as a multi-channel input to synthesize the visible image given the corresponding polarimetric signatures. Although these methods have yielded improvemen…
Various and different methods can be used to produce high-resolution multispectral images from high-resolution panchromatic image (PAN) and low-resolution multispectral images (MS), mostly on the pixel level. The Quality of image fusion is an essential determinant of the value of processing images fusion for many applications. Spatial and spectral qualities are the two important indexes that used to evaluate the quality of any fused image. However, the jury is still out of fused image's benefits if it compared with its original images. In addition, there is a lack of measures for assessing the…
SMLNet: A SPD Manifold Learning Network for Infrared and Visible Image Fusion
School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院）；Jiangnan University（江南大学）；Centre for Vision, Speech and Signal Processing（视觉、语音与信号处理中心）；University of Surrey（Surrey大学）
A review of deep learning-based information fusion techniques for multimodal medical image classification
Inserm（法国国家健康与医学研究院）；University of Western Brittany（西布列塔尼大学）；IMT Atlantique（大西洋电信学院）；Sorbonne University（索邦大学）；Lariboisière Hospital（拉里布瓦西耶尔医院）；AP-HP（巴黎公共医疗集团）；Paris Cité University（巴黎城市大学）；CHRU Brest（布雷斯特大学中心医院）；LaTIM UMR 1101（LaTIM联合研究单位1101）
An increasing number of laser powder bed fusion machines use off-axis infrared cameras to improve online monitoring and data-driven control capabilities. However, there is still a severe lack of algorithmic solutions to properly process the infrared images from these cameras that has led to several key limitations: a lack of online monitoring capabilities for the laser tracks, insufficient pre-processing of the infrared images for data-driven methods, and large memory requirements for storing the infrared images. To address these limitations, we study over 30 segmentation algorithms that segme…
Bu-Ali Sina University（布阿里·西纳大学）；Alzahra University（扎赫拉大学）；University of Birjand（比尔詹德大学）
In this paper, a novel objective evaluation metric for image fusion is presented. Remarkable and attractive points of the proposed metric are that it has no parameter, the result is probability in the range of [0, 1] and it is free from illumination dependence. This metric is easy to implement and the result is computed in four steps: (1) Smoothing the images using Gaussian filter. (2) Transforming images to a vector field using Del operator. (3) Computing the normal distribution function (μ,σ) for each corresponding pixel, and converting to the standard normal distribution function. (4) Compu…
Cloud Removal With PolSAR-Optical Data Fusion Using A Two-Flow Residual Network
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；International Research Center of Big Data for Sustainable Development Goals（国际可持续发展大数据研究中心）；College of Resources and Environment, University of Chinese Academy of Sciences（中国科学院大学资源与环境学院）
Institute for Anthropomatics and Robotics（拟人学与机器人学研究所）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Visual recognition inside the vehicle cabin leads to safer driving and more intuitive human-vehicle interaction but such systems face substantial obstacles as they need to capture different granularities of driver behaviour while dealing with highly limited body visibility and changing illumination. Multimodal recognition mitigates a number of such issues: prediction outcomes of different sensors complement each other due to different modality-specific strengths and weaknesses. While several late fusion methods have been considered in previously published frameworks, they constantly feature di…
College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·哈利法大学科学与工程学院）；College of Health and Life Sciences, Hamad Bin Khalifa University（哈马德·本·哈利法大学健康与生命科学学院）
Healthcare data are inherently multimodal, including electronic health records (EHR), medical images, and multi-omics data. Combining these multimodal data sources contributes to a better understanding of human health and provides optimal personalized healthcare. Advances in artificial intelligence (AI) technologies, particularly machine learning (ML), enable the fusion of these different data modalities to provide multimodal insights. To this end, in this scoping review, we focus on synthesizing and analyzing the literature that uses AI techniques to fuse multimodal medical data for different…
