arXiv 2506.227362025-06-28cs.CV

UniFuse：面向多种退化与失配情况的多模态医学图像融合统一一体化框架

UniFuse: A Unified All-in-One Framework for Multi-Modal Medical Image Fusion Under Diverse Degradations and Misalignments

Kunming University of Science and Technology（昆明理工大学）；Harbin Institute of Technology at Shenzhen（哈尔滨工业大学深圳研究院）；Hefei University of Technology（合肥工业大学）

Dayong Su, Yafei Zhang, Huafeng Li, Jinxing Li, Yu Liu

备注：: Accepted by ICCV2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对医学图像失配与退化问题，提出统一框架UniFuse，通过退化感知提示学习、Omni Unified Feature Representation及基于LoRA的ALSN模块，在单一框架内联合实现跨模态对齐、恢复与融合，性能显著优于现有方法。

摘要：

当前的多模态医学图像融合通常假设源图像质量高且在像素级完美对齐。其有效性严重依赖这些条件，在处理失配或退化的医学图像时往往效果变差。为此，我们提出UniFuse，一个通用融合框架。通过嵌入退化感知提示学习模块，UniFuse无缝整合输入图像的多方向信息，并将跨模态对齐与恢复关联，在统一框架内实现两项任务的联合优化。此外，我们设计了Omni Unified Feature Representation方案，利用Spatial Mamba编码多方向特征并缓解特征对齐中的模态差异。为在All-in-One配置下实现同时恢复与融合，我们提出Universal Feature Restoration & Fusion模块，其中引入基于LoRA原理的Adaptive LoRA Synergistic Network（ALSN）。利用ALSN的自适应特征表示及退化类型引导，我们在单阶段框架内实现联合恢复与融合。与分阶段方法相比，UniFuse在单一框架内统一了对齐、恢复与融合。在多个数据集上的实验结果表明了该方法的有效性及相比现有方法的显著优势。

英文摘要：

Current multimodal medical image fusion typically assumes that source images are of high quality and perfectly aligned at the pixel level. Its effectiveness heavily relies on these conditions and often deteriorates when handling misaligned or degraded medical images. To address this, we propose UniFuse, a general fusion framework. By embedding a degradation-aware prompt learning module, UniFuse seamlessly integrates multi-directional information from input images and correlates cross-modal alignment with restoration, enabling joint optimization of both tasks within a unified framework. Additionally, we design an Omni Unified Feature Representation scheme, which leverages Spatial Mamba to encode multi-directional features and mitigate modality differences in feature alignment. To enable simultaneous restoration and fusion within an All-in-One configuration, we propose a Universal Feature Restoration & Fusion module, incorporating the Adaptive LoRA Synergistic Network (ALSN) based on LoRA principles. By leveraging ALSN's adaptive feature representation along with degradation-type guidance, we enable joint restoration and fusion within a single-stage framework. Compared to staged approaches, UniFuse unifies alignment, restoration, and fusion within a single framework. Experimental results across multiple datasets demonstrate the method's effectiveness and significant advantages over existing approaches.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。