IT-TextFusion：用于感知退化的图像融合的迭代文本-图像交互与文本引导残差细化

中文摘要 AI 辅助

文本引导的图像融合近年来已成为整合多模态信息、实现灵活且面向任务的融合控制的有效范式。然而，现有的文本引导融合方法往往依赖浅层的语义-视觉交互与有限的注意力机制，这限制了它们稳健处理复杂退化以及充分利用文本引导的能力。本文提出一种迭代文本引导图像融合框架，该框架在多个融合与细化阶段融入文本条件特征交互。所提方法整合最深层的交叉注意力（Cross-Attention）、多尺度交叉门融合（Cross-Gate Fusion）以及阶段特定的文本条件调制，使全局文本嵌入能够调控分层特征融合与残差细化。通过在分层解码器与细化阶段反复注入池化后的文本嵌入，该框架在保留可见光与红外模态互补信息的同时，提供感知退化的全局语义调控。在多个基准数据集上的实验表明，所提方法提升了多项信息保留与感知质量指标，同时在部分数据集上呈现出指标依赖的权衡。

英文摘要

Text-guided image fusion has recently emerged as an effective paradigm for integrating multi-modal information while enabling flexible and task-oriented fusion control. However, existing text-guided fusion methods often rely on shallow semantic-visual interaction and limited attention mechanisms, which restrict their ability to robustly handle complex degradations and fully exploit textual guidance. In this paper, we propose an iterative text-guided image fusion framework that incorporates text-conditioned feature interaction across multiple fusion and refinement stages. The proposed method integrates deepest-level Cross-Attention, multi-scale Cross-Gate Fusion, and stage-specific text-conditioned modulation, allowing the global text embedding to condition hierarchical feature fusion and residual refinement. By repeatedly injecting the pooled text embedding across hierarchical decoder and refinement stages, the proposed framework provides degradation-aware global semantic conditioning while preserving complementary information from the visible and infrared modalities. Experiments on several benchmark datasets show that the proposed method improves several information-preservation and perceptual-quality metrics, while exhibiting metric-dependent trade-offs on some datasets.

发表机构

• School of Automation, Southeast University（东南大学自动化学院）
• Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统研究所）

机构由 AI 辅助整理，请以论文原文为准。