Multi-focus image fusion (MFIF) addresses the depth-of-field (DOF) limitations of optical lenses, where only objects within a specific range appear sharp. Although traditional and deep learning methods have advanced the field, challenges persist, including limited training data, domain gaps from synthetic datasets, and difficulties with regions lacking information. We propose VAEEDOF, a novel MFIF method that uses a distilled variational autoencoder for high-fidelity, efficient image reconstruction. Our fusion module processes up to seven images simultaneously, enabling robust fusion across diverse focus points. To address data scarcity, we introduce MattingMFIF, a new syntetic 4K dataset, simulating realistic DOF effects from real photographs. Our method achieves state-of-the-art results, generating seamless artifact-free fused images and bridging the gap between synthetic and real-world scenarios, offering a significant step forward in addressing complex MFIF challenges. The code, and weights are available here:
When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion
College of Computer Science and Engineering, Dalian Minzu University（大连民族大学计算机科学与工程学院）；School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；Department of Automation, Tsinghua University（清华大学自动化系）
$\textrm{A}^{\textrm{2}}$RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion
School of Computer and Communication Engineering, University of Science and Technology Beijing（北京科技大学计算机与通信工程学院）；School of Software Technology, Dalian University of Technology（大连理工大学软件学院）
The most significant problem may be undesirable effects for the spectral signatures of fused images as well as the benefits of using fused images mostly compared to their source images were acquired at the same time by one sensor. They may or may not be suitable for the fusion of other images. It becomes therefore increasingly important to investigate techniques that allow multi-sensor, multi-date image fusion to make final conclusions can be drawn on the most suitable method of fusion. So, In this study we present a new method Segmentation Fusion method (SF) for remotely sensed images is pres…
Cross Paradigm Representation and Alignment Transformer for Image Deraining
Nanjing Agricultural University（南京农业大学）；Soochow University（苏州大学）；Xiangtan University（湘潭大学）；Shanghai University（上海大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Westlake University（西湖大学）
IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统研究所）
One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion
School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机学院）；Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey（萨里大学视觉、语音与信号处理中心）
With the advent of deep learning methods replacing the ISP in transforming sensor RAW readings into RGB images, numerous methodologies solidified into real-life applications. Equally potent is the task of inverting this process which will have applications in enhancing computational photography tasks that are conducted in the RAW domain, addressing lack of available RAW data while reaping from the benefits of performing tasks directly on sensor readings. This paper's proposed methodology is a state-of-the-art solution to the task of RAW reconstruction, and the multi-step refinement process int…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学综合业务网理论及关键技术国家重点实验室）；School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；University of Pittsburgh（匹兹堡大学）
Instance segmentation is an important task for biomedical and biological image analysis. Due to the complicated background components, the high variability of object appearances, numerous overlapping objects, and ambiguous object boundaries, this task still remains challenging. Recently, deep learning based methods have been widely employed to solve these problems and can be categorized into proposal-free and proposal-based methods. However, both proposal-free and proposal-based methods suffer from information loss, as they focus on either global-level semantic or local-level instance features…
A Pseudo Global Fusion Paradigm-Based Cross-View Network for LiDAR-Based Place Recognition
School of Electronics and Information Engineering, and Xingzhi College, South China Normal University（电子信息工程学院和星智学院，华南师范大学）；School of Data Science and Engineering, and Xingzhi College, South China Normal University（数据科学与工程学院和星智学院，华南师范大学）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）；College of Electronics & Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University（电子与信息工程学院，上海智能自主系统研究所，智能自主系统国家重点实验室，智能自主系统前沿科学中心，同济大学）；Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（电子与计算机工程系，香港科技大学）
University of Trento（特伦托大学）；Huawei Technologies Ireland（华为技术爱尔兰）
We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images. In this way, we can exploit multiple, possibly complementary images of the same person which are usually available at training and at testing time. The solution we propose is mainly based on a local attention mechanism which selects relevant information from different source image regions, avoiding the necessity to build specific generators for each specific cardinality of X. The empirical evaluation of our method shows the p…
DGFusion: Dual-guided Fusion for Robust Multi-Modal 3D Object Detection
School of Computer Science and Technology, Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University（计算机科学与技术学院、交通数据挖掘与具身智能北京市重点实验室、北京交通大学）；State Key Laboratory of Internet of Things for Smart City and Department of Electrome chanical Engineering, University of Macau（智能城市物联网国家重点实验室、澳门大学机电工程系）；Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University（智能城市感知与计算福建省重点实验室、厦门大学）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（机械与航空航天工程学院、南洋理工大学）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（机器人系统国家重点实验室、哈尔滨工业大学）
Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation
School of Nursing, The Hong Kong Polytechnic University（香港理工大学护理学院）；School of Software Engineering, South China University of Technology（华南理工大学软件学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）
针对 EHR 与 CXR 采集不同步导致旧胸片影响临床预测的问题，提出 DDL-CXR，利用既往 CXR 和 EHR 时间序列条件化潜在扩散模型生成最新患者特异性表示，并在 MIMIC 上验证其性能优势。
Select-and-Combine (SAC): A Novel Multi-Stereo Depth Fusion Algorithm for Point Cloud Generation via Efficient Local Markov Netlets
The Ohio State University（俄亥俄州立大学）；Geospatial Data Analytics lab（地理空间数据分析实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Translational Data Analytics Institute（转化数据分析研究所）
IIIT Delhi（德里印度理工学院）；Michigan State University（密歇根州立大学）
The performance of a biometric system that relies on a single biometric modality (e.g., fingerprints only) is often stymied by various factors such as poor data quality or limited scalability. Multibiometric systems utilize the principle of fusion to combine information from multiple sources in order to improve recognition accuracy whilst addressing some of the limitations of single-biometric systems. The past two decades have witnessed the development of a large number of biometric fusion schemes. This paper presents an overview of biometric fusion with specific focus on three questions: what…
Jiangnan University（江南大学）；University of Surrey（萨里大学）
The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on a lightweight transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relat…
Object Tracking is one important problem in computer vision and surveillance system. The existing models mainly exploit the single-view feature (i.e. color, texture, shape) to solve the problem, failing to describe the objects comprehensively. In this paper, we solve the problem from multi-view perspective by leveraging multi-view complementary and latent information, so as to be robust to the partial occlusion and background clutter especially when the objects are similar to the target, meanwhile addressing tracking drift. However, one big problem is that multi-view fusion strategy can inevit…
Due to the imbalanced and limited data, semi-supervised medical image segmentation methods often fail to produce superior performance for some specific tailed classes. Inadequate training for those particular classes could introduce more noise to the generated pseudo labels, affecting overall learning. To alleviate this shortcoming and identify the under-performing classes, we propose maintaining a confidence array that records class-wise performance during training. A fuzzy fusion of these confidence scores is proposed to adaptively prioritize individual confidence metrics in every sample rat…
University of Sassari（萨萨里大学）；Jadavpur University（贾达夫普尔大学）；I.I.T Kanpur（印度理工学院坎普尔分校）
Faces are highly deformable objects which may easily change their appearance over time. Not all face areas are subject to the same variability. Therefore decoupling the information from independent areas of the face is of paramount importance to improve the robustness of any face recognition technique. This paper presents a robust face recognition technique based on the extraction and matching of SIFT features related to independent face areas. Both a global and local (as recognition from parts) matching strategy is proposed. The local strategy is based on matching individual salient facial SI…
Talk2PC: Enhancing 3D Visual Grounding through LiDAR and Radar Point Clouds Fusion for Autonomous Driving
Thrust of Artificial Intelligence, Hong Kong University of Science and Technology (Guangzhou), China（香港理工大学（广州）人工智能研究所）；Mononai AI, Sweden（蒙诺人工智能）；College of Computer Science and Technology, China University of Petroleum (East China)（中国石油大学（华东）计算机科学与技术学院）；School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学先进科技学院）；Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；School of Automation, Southeast University（东南大学自动化学院）；College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；School of Transportation, Southeast University（东南大学交通运输学院）
Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration
Beijing Institute of Technology（北京理工大学）；Research and Development Center of Agricultural Bank of China（中国农业银行研发中心）；Peking University（北京大学）；Beijing Normal University（北京师范大学）
University of Science and Technology of China（中国科学技术大学）；Sichuan University（四川大学）
RGB-D salient object detection (SOD) recently has attracted increasing research interest and many deep learning methods based on encoder-decoder architectures have emerged. However, most existing RGB-D SOD models conduct feature fusion either in the single encoder or the decoder stage, which hardly guarantees sufficient cross-modal fusion ability. In this paper, we make the first attempt in addressing RGB-D SOD through 3D convolutional neural networks. The proposed model, named RD3D, aims at pre-fusion in the encoder stage and in-depth fusion in the decoder stage to effectively promote the ful…
Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection
Anhui University（安徽大学）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；Institute of Physical Science and Information Technology, Anhui University（安徽大学物理科学与信息技术学院）
Recent learning-based image fusion methods have marked numerous progress in pre-registered multi-modality data, but suffered serious ghosts dealing with misaligned multi-modality data, due to the spatial deformation and the difficulty narrowing cross-modality discrepancy. To overcome the obstacles, in this paper, we present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion (IVIF). Specifically, we propose a Cross-modality Perceptual Style Transfer Network (CPSTN) to generate a pseudo infrared image taking a visible image as i…
University of British Columbia（不列颠哥伦比亚大学）；BC Cancer Research Institute（不列颠哥伦比亚癌症研究所）；Geneva University Hospital（日内瓦大学医院）
Radiomics features extract quantitative information from medical images, towards the derivation of biomarkers for clinical tasks, such as diagnosis, prognosis, or treatment response assessment. Different image discretization parameters (e.g. bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels can be used to generate radiomics features and ultimately signatures. Commonly, only one set of parameters is used; resulting in only one value or flavour for a given RF. We propose tensor radiomics (TR) where tensors of features calculated with multiple …
Although much significant progress has been made in the research field of object detection with deep learning, there still exists a challenging task for the objects with small size, which is notably pronounced in UAV-captured images. Addressing these issues, it is a critical need to explore the feature extraction methods that can extract more sufficient feature information of small objects. In this paper, we propose a novel method called Dense Multiscale Feature Fusion Pyramid Networks(DMFFPN), which is aimed at obtaining rich features as much as possible, improving the information propagation…
Combining information from multi-view images is crucial to improve the performance and robustness of automated methods for disease diagnosis. However, due to the non-alignment characteristics of multi-view images, building correlation and data fusion across views largely remain an open problem. In this study, we present TransFusion, a Transformer-based architecture to merge divergent multi-view imaging information using convolutional layers and powerful attention mechanisms. In particular, the Divergent Fusion Attention (DiFA) module is proposed for rich cross-view context modeling and semanti…
Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles
College of Engineering, Center for Environmental Research and Technology, University of California at Riverside（工程学院、环境研究与技术中心、加州大学河滨分校）；School of Transportation and Logistics, Southwest Jiaotong University（交通运输与物流学院、西南交通大学）
Feature fusion plays a crucial role in unconstrained face recognition where inputs (probes) comprise of a set of $N$ low quality images whose individual qualities vary. Advances in attention and recurrent modules have led to feature fusion that can model the relationship among the images in the input set. However, attention mechanisms cannot scale to large $N$ due to their quadratic complexity and recurrent modules suffer from input order sensitivity. We propose a two-stage feature fusion paradigm, Cluster and Aggregate, that can both scale to large $N$ and maintain the ability to perform sequ…
