arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02754cs.CV

无需训练即可清晰感知:减轻多模态大语言模型在遥感中的幻觉

Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

School of Computer Science, Wuhan University, China(武汉大学计算机学院);Zhongguancun Academy, China(中关村学院);School of Computer Science, Chongqing University, China(重庆大学计算机学院)

Yi Liu, Jing Zhang, Di Wang, Xiaoyu Tian, Haonan Guo, Bo Du

AI总结:

RADAR通过利用多模态大语言模型的内在注意力,无需训练即可减少遥感视觉问答中的事实性和逻辑性幻觉。

摘要:

多模态大语言模型(MLLMs)在遥感视觉问答(RS-VQA)中表现出显著的幻觉问题,主要由大规模场景中的视觉定位失败或对细粒度小目标的误判引起。为了系统分析这些问题,我们引入RSHBench,一个基于协议的基准测试,用于对事实性和逻辑性幻觉进行细粒度诊断。为了缓解由定位引起的事实性幻觉,我们进一步提出相对注意力驱动的主动推理(RADAR),一种无需训练的推理方法,利用MLLMs内在的注意力来指导测试时的逐步定位和细粒度局部推理。在多样化的MLLMs上进行的广泛实验表明,RADAR一致提高了RS-VQA性能,并减少了事实性和逻辑性幻觉。代码和数据将在:https://github.com/MiliLab/RADAR上公开。

英文摘要:

Multimodal large language models (MLLMs) suffer from pronounced hallucinations in remote sensing visual question-answering (RS-VQA), primarily caused by visual grounding failures in large-scale scenes or misinterpretation of fine-grained small targets. To systematically analyze these issues, we introduce RSHBench, a protocol-based benchmark for fine-grained diagnosis of factual and logical hallucinations. To mitigate grounding-induced factual hallucinations, we further propose Relative Attention-Driven Actively Reasoning (RADAR), a training-free inference method that leverages intrinsic attention in MLLMs to guide progressive localization and fine-grained local reasoning at test time. Extensive experiments across diverse MLLMs demonstrate that RADAR consistently improves RS-VQA performance and reduces both factual and logical hallucinations. Code and data will be publicly available at: https://github.com/MiliLab/RADAR

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑