RoES：面向多模态图像的旋转等变选择性频率融合
RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images
- Zhongnan University of Economics and Law（中南财经政法大学）
- City University of Macau（澳门城市大学）
- University of Illinois Chicago（伊利诺伊大学芝加哥分校）
- Hubei University（湖北大学）
- Xi’an High-tech Research Institute（西安高新技术研究所）
- China University of Geosciences (Wuhan)（中国地质大学（武汉））
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对多模态图像融合中旋转等变约束与频率特性冲突的问题，提出RoES网络，通过动态解耦低频与高频并分别采用等变Mamba和频谱注意力处理，在融合质量与下游检测上达到最先进性能。
AI中文摘要：
红外-可见光图像融合通过整合可见光传感器中的互补纹理细节与红外系统中的热特征，促进了鲁棒的多模态感知。由于该任务本质上具有不适定性，现有方法严重依赖结构先验，但通常对所有特征均匀地强制施加旋转等变性。这种整体性方法忽略了一个关键区别：低频共享结构严格遵循等变约束，而高频模态特定细节则需要更大的灵活性以保留独特信息。为弥合这一差距，我们提出了RoES，一种旋转等变选择性频率融合网络。我们不采用静态分解，而是引入一个可训练的旋转增强更新器/预测器模块，以动态解耦低频和高频分量。所得表示随后通过一个针对频谱一致性定制的双分支融合模块进行处理。具体而言，采用旋转等变Mamba来捕获低频域中的长程结构依赖性，而基于极坐标频谱注意力的双傅里叶块在显式低频引导下细化高频细节。大量实验表明，RoES在融合质量和下游目标检测方面均持续达到最先进性能，通过调和频率选择性特征与等变约束，为多模态融合建立了鲁棒解决方案。源代码可在该https URL获取。
英文摘要：
Infrared-visible image fusion facilitates robust multimodal perception by integrating complementary textural nuances from visible sensors with thermal signatures from infrared systems. Due to the task's inherently ill-posed nature, existing methods heavily rely on structural priors but typically enforce rotation equivariance uniformly across all features. Such a holistic approach overlooks a critical distinction where low-frequency shared structures strictly adhere to equivariant constraints while high-frequency modality-specific details require greater flexibility to preserve unique information. To bridge this gap, we propose RoES, a Rotational Equivariant Selective-frequency fusion network. Instead of employing static decomposition, we introduce a trainable rotation-enhanced updater/predictor module to dynamically decouple low- and high-frequency components. The resulting representations are then processed through a dual-branch fusion module tailored for spectral consistency. Specifically, a rotation-equivariant Mamba is employed to capture long-range structural dependencies in the low-frequency domain, while a polar spectral attention-based Dual-Fourier block refines high-frequency details under explicit low-frequency guidance. Extensive experiments demonstrate that RoES consistently achieves state-of-the-art performance in both fusion quality and downstream object detection, establishing a robust solution for multimodal fusion by reconciling frequency-selective features with equivariant constraints. The source code is available at https://github.com/BryceLosky/RoES-Fusion.
