发表机构
Hefei University of Technology; Kunming University of Science and Technology（合肥工业大学; 昆明理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出CRT-HMAR框架，通过因果需求追踪与分层多智能体调控，实现红外-可见光图像融合对未知开放任务的泛化，实验验证其提升泛化并保持已知任务性能。
AI 中文摘要
红外与可见光（IR-VIS）图像融合将互补的多模态信息整合到单一融合图像中，以支持下游视觉任务。然而，现有方法通常针对固定任务集中的已知任务进行定制，难以泛化到未知任务，这限制了它们在真实世界开放任务场景中的适用性。为解决这一问题，本文提出CRT-HMAR，一种面向开放任务感知的IR-VIS图像融合的因果需求追踪引导的分层多智能体调控框架。CRT-HMAR引入因果需求追踪任务定位机制，该机制主动干预关键图像信息并观察任务网络响应变化，从而将任务特定的语义偏好映射为图像级别的因果需求图。基于这些图，需求分析智能体聚合任务特定的需求知识，以自适应地引导需求定制的图像融合。此外，CRT-HMAR整合了历史分析多目标平衡和任务级校正冲突缓解机制，共同构建了“需求解释-任务平衡-冲突缓解”的分层调控链。通过多个协作智能体，CRT-HMAR动态调控关键过程，包括开放任务需求建模、多任务平衡优化和梯度冲突缓解。在涉及五个下游任务的开放任务场景上的大量实验表明，CRT-HMAR显著提高了对未知任务的泛化能力，同时保持了已知任务的性能和平衡。总体而言，CRT-HMAR将IR-VIS图像融合从面向任务的建模转向面向需求的建模，推动其从封闭任务设置扩展到真实世界的开放任务场景。
英文摘要
Infrared and visible (IR-VIS) image fusion integrates complementary multimodal information into a single fused image to support downstream vision tasks. However, existing methods are typically tailored to seen tasks within a fixed task set and struggle to generalize to unseen tasks, which restricts their applicability in real-world open-task scenarios. To address this issue, this paper proposes CRT-HMAR, a Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation Framework for open-task-aware IR-VIS image fusion. CRT-HMAR introduces a Causal Requirement Tracing Task Localization mechanism, which actively intervenes in key image information and observes task-network response variations to map task-specific semantic preferences into image-level causal requirement maps. Based on these maps, a requirement analysis agent aggregates task-specific requirement knowledge to adaptively guide requirement-customized image fusion. Moreover, CRT-HMAR incorporates History-Analysis Multi-Objective Balancing and Task-Level-Correction Conflict Mitigation mechanisms, jointly constructing a hierarchical regulation chain of "requirement interpretation - task balancing - conflict mitigation". Through multiple collaborative agents, CRT-HMAR dynamically regulates key processes including open-task requirement modeling, multi-task balanced optimization, and gradient conflict mitigation. Extensive experiments on open-task scenarios involving five downstream tasks demonstrate that CRT-HMAR significantly improves generalization to unseen tasks while maintaining the performance and balance of seen tasks. Overall, CRT-HMAR shifts IR-VIS image fusion from task-oriented modeling toward requirement-oriented modeling, promoting its extension from closed-task settings to real-world open-task scenarios.
Comments17 pages, 11 figures
