思考并回答ME:基准测试和探索遥感中的多实体推理 grounding
Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing
AI总结:
本文提出ME-RSRG基准数据集,通过Entity-Aware Reasoning框架提升遥感多实体推理能力,验证了该框架的有效性。
摘要:
近期推理语言模型和具有可验证奖励的强化学习的进展显著增强了多步推理能力。这一进步促使推理范式扩展到遥感视觉 grounding 任务。然而,现有遥感 grounding 方法仍主要局限于感知层面匹配和单实体公式化,限制了显式推理和实体间建模的作用。为解决这一挑战,我们引入了一个新的基准数据集用于遥感中的多实体推理 grounding(ME-RSRG)。基于ME-RSRG,我们将遥感 grounding 重新公式化为一个多实体推理任务,并提出一个基于视觉-语言基础模型的实体感知推理(EAR)框架。EAR生成结构化的推理轨迹和主体-对象 grounding 输出。它采用监督微调进行冷启动初始化,并进一步通过实体感知奖励驱动的组相对策略优化(GRPO)进行优化。在ME-RSRG上的广泛实验验证了多实体推理的挑战,并验证了所提EAR框架的有效性。我们的数据集、代码和模型将在https://github.com/CV-ShuchangLyu/ME-RSRG上提供。
英文摘要:
Recent advances in reasoning language models and reinforcement learning with verifiable rewards have significantly enhanced multi-step reasoning capabilities. This progress motivates the extension of reasoning paradigms to remote sensing visual grounding task. However, existing remote sensing grounding methods remain largely confined to perception-level matching and single-entity formulations, limiting the role of explicit reasoning and inter-entity modeling. To address this challenge, we introduce a new benchmark dataset for Multi-Entity Reasoning Grounding in Remote Sensing (ME-RSRG). Based on ME-RSRG, we reformulate remote sensing grounding as a multi-entity reasoning task and propose an Entity-Aware Reasoning (EAR) framework built upon visual-linguistic foundation models. EAR generates structured reasoning traces and subject-object grounding outputs. It adopts supervised fine-tuning for cold-start initialization and is further optimized via entity-aware reward-driven Group Relative Policy Optimization (GRPO). Extensive experiments on ME-RSRG demonstrate the challenges of multi-entity reasoning and verify the effectiveness of our proposed EAR framework. Our dataset, code, and models will be available at https://github.com/CV-ShuchangLyu/ME-RSRG.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。