arXiv 2512.096632025-12-10cs.CV

IF-Bench：基于生成视觉提示的多模态大语言模型在红外图像上的基准测试与增强

IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting

MAIS, Institute of Automation（自动化研究所信息处理中心）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；Research Center of Aerospace Information, Institute of Automation（自动化研究所航空信息研究中心）

Tao Zhang, Yuyang Hong, Yang Xia, Kun Ding, Zeyu Zhang, Ying Wang, Shiming Xiang, Chunhong Pan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

IF-Bench通过生成视觉提示方法提升多模态大语言模型对红外图像的理解能力，提供首个高质量基准测试及实验验证。

摘要：

近年来，多模态大语言模型（MLLMs）的进步在各种基准测试中取得了显著进展。然而，其在理解红外图像方面的能力仍未被探索。为填补这一空白，我们引入了IF-Bench，这是首个高质量的基准测试，用于评估多模态对红外图像的理解能力。IF-Bench包含499张来自23个红外数据集的图像和680对精心挑选的视觉问题-答案对，涵盖图像理解的10个关键维度。基于此基准测试，我们系统地评估了超过40个开源和闭源的MLLMs，采用循环评估、双语评估和混合判断策略来提高结果的可靠性。我们的分析揭示了模型规模、架构和推理范式如何影响红外图像的理解，为该领域提供了有价值的见解。此外，我们提出了一种无需训练的生成视觉提示（GenViP）方法，利用先进的图像编辑模型将红外图像转换为在语义和空间上对齐的RGB图像，从而缓解领域分布偏移。广泛的实验表明，我们的方法在各种MLLMs上都产生了显著的性能提升。该基准测试和代码可在https://github.com/casiatao/IF-Bench上获取。

英文摘要：

Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images remains unexplored. To address this gap, we introduce IF-Bench, the first high-quality benchmark designed for evaluating multimodal understanding of infrared images. IF-Bench consists of 499 images sourced from 23 infrared datasets and 680 carefully curated visual question-answer pairs, covering 10 essential dimensions of image understanding. Based on this benchmark, we systematically evaluate over 40 open-source and closed-source MLLMs, employing cyclic evaluation, bilingual assessment, and hybrid judgment strategies to enhance the reliability of the results. Our analysis reveals how model scale, architecture, and inference paradigms affect infrared image comprehension, providing valuable insights for this area. Furthermore, we propose a training-free generative visual prompting (GenViP) method, which leverages advanced image editing models to translate infrared images into semantically and spatially aligned RGB counterparts, thereby mitigating domain distribution shifts. Extensive experiments demonstrate that our method consistently yields significant performance improvements across a wide range of MLLMs. The benchmark and code are available at https://github.com/casiatao/IF-Bench.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。