WIFE-Fusion：面向多模态图像融合的小波感知的 Intra-inter 频率增强
WIFE-Fusion:Wavelet-aware Intra-inter Frequency Enhancement for Multi-model Image Fusion
AI总结：
本文提出 WIFE-Fusion 多模态图像融合框架，通过频域内自注意力和频间交互增强频域特征提取与聚合，在多个任务和数据集上优于现有专门及统一融合方法。
摘要：
多模态图像融合能够有效聚合来自不同模态的信息，融合图像在视觉系统中发挥着关键作用。然而，现有方法往往忽视频域特征探索和交互关系。本文提出小波感知的 Intra-inter 频率增强融合（WIFE-Fusion），这是一个基于频域分量交互的多模态图像融合框架。其核心创新包括：Intra-Frequency Self-Attention（IFSA），通过交互式自注意力机制利用固有的跨模态相关性和互补性来提取丰富的频域特征；以及 Inter-Frequency Interaction（IFI），通过跨模态异构频域分量之间的组合交互来增强丰富特征并过滤潜在特征。这些过程实现了精确的源特征提取以及特征提取-聚合的统一建模。在三个多模态融合任务、五个数据集上的大量实验表明，WIFE-Fusion 优于当前专门的融合方法和统一融合方法。我们的代码可在 https://github.com/Lmmh058/WIFE-Fusion 获取。
英文摘要：
Multimodal image fusion effectively aggregates information from diverse modalities, with fused images playing a crucial role in vision systems. However, existing methods often neglect frequency-domain feature exploration and interactive relationships. In this paper, we propose wavelet-aware Intra-inter Frequency Enhancement Fusion (WIFE-Fusion), a multimodal image fusion framework based on frequency-domain components interactions. Its core innovations include: Intra-Frequency Self-Attention (IFSA) that leverages inherent cross-modal correlations and complementarity through interactive self-attention mechanisms to extract enriched frequency-domain features, and Inter-Frequency Interaction (IFI) that enhances enriched features and filters latent features via combinatorial interactions between heterogeneous frequency-domain components across modalities. These processes achieve precise source feature extraction and unified modeling of feature extraction-aggregation. Extensive experiments on five datasets across three multimodal fusion tasks demonstrate WIFE-Fusion's superiority over current specialized and unified fusion methods. Our code is available at https://github.com/Lmmh058/WIFE-Fusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
