评估与增强遥感多模态大语言模型的否定理解能力
Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs
AI总结：
提出RS-Neg基准评估遥感MLLMs的否定理解，并设计NeFo方法通过测试时学习利用约5%未标注样本显著提升模型性能。
摘要：
多模态大语言模型（MLLMs）在各种遥感（RS）任务中取得了显著成功。然而，它们理解否定的能力仍未得到充分探索，限制了在现实应用中的部署，其中模型必须明确识别什么是错误的或不存在的，例如，应急响应人员需要定位非洪水路线进行疏散。为了全面研究这一局限性，我们引入了RS-Neg，这是第一个从区域级到场景级任务评估否定理解的基准。具体来说，我们为遥感图像设计了一个自动数据生成流程，使用LLMs合成多样化的否定查询，并引入了一个动态视觉焦点模块进行验证。我们的评估表明，先进的遥感MLLMs在否定理解上存在困难，表现出幻觉和显著的性能下降。为了弥补这一差距，我们提出了NeFo，一种新颖的测试时学习方法，将否定的逻辑角色明确纳入模型优化。值得注意的是，使用约5%的未标注测试样本，NeFo显著提升了模型的否定理解能力，并展现出对未见任务的强泛化能力。代码和数据将在接收后发布。
英文摘要：
Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in various Remote Sensing (RS) tasks. However, their ability to comprehend negation remains underexplored, limiting deployment in real-world applications where models must explicitly identify what is false or absent, e.g., emergency responders need to locate non-flooded routes for evacuation. To comprehensively study this limitation, we introduce RS-Neg, the first benchmark to evaluate negation understanding across region-level to scene-level tasks. Specifically, we design an automated data generation pipeline for RS imagery, using LLMs to synthesize diverse negation queries, and introduce a dynamic visual focus module for verification. Our evaluation reveals that advanced RS MLLMs struggle with negation, exhibiting hallucinations and substantial performance degradation. To close this gap, we propose NeFo, a novel test-time learning method that explicitly incorporates the logical role of negation into the model optimization. Remarkably, using about 5\% unlabeled test samples, NeFo significantly improves the negation understanding of models and shows strong generalization to unseen tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
