RemoteAgent: 通过基于强化学习的代理多模态大语言模型弥合模糊人类意图与遥感观测之间的鸿沟
RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs
AI总结：
本文提出RemoteAgent框架，通过VagueEO数据集和强化学习微调，使多模态大语言模型能直接解决图像和稀疏区域任务，并通过模型上下文协议智能协调专用工具进行密集预测，提升遥感任务的性能。
摘要：
遥感（EO）系统本质上是为领域专家设计的，这些专家通常通过模糊的自然语言而非精确的机器友好指令表达需求。根据具体应用场景，这些模糊查询可能需要从整体图像解释到细粒度像素预测的不同层次的视觉精度。因此，一个实用的EO人工智能系统必须弥合模糊人类查询与适当多粒度视觉分析任务之间的鸿沟。尽管多模态大语言模型（MLLMs）表现出强大的语义理解能力，但其基于文本的输出格式本质上不适合密集、精度要求高的空间预测。现有的代理框架通过将任务委托给外部工具来解决这一限制，但随意调用工具在计算上效率低下且未充分利用MLLM的原生能力。为此，我们提出RemoteAgent，一种代理框架，战略性地尊重MLLM的内在能力边界。为了使该框架能够理解真实用户意图，我们构建了VagueEO，一个以人类为中心的指令数据集，将EO任务与模拟的模糊自然语言查询配对。通过利用VagueEO进行强化学习微调，我们将MLLM对齐为一个稳健的认知核心，能够直接解决图像和稀疏区域级别的任务。因此，RemoteAgent在内部处理合适的任务，同时通过模型上下文协议智能协调专用工具进行密集预测。广泛的实验表明，RemoteAgent在实现稳健的意图识别能力的同时，在多样化的EO任务中表现出高度竞争性的性能。
英文摘要：
Earth Observation (EO) systems are essentially designed to support domain experts who often express their requirements through vague natural language rather than precise, machine-friendly instructions. Depending on the specific application scenario, these vague queries can demand vastly different levels of visual precision. Consequently, a practical EO AI system must bridge the gap between ambiguous human queries and the appropriate multi-granularity visual analysis tasks, ranging from holistic image interpretation to fine-grained pixel-wise predictions. While Multi-modal Large Language Models (MLLMs) demonstrate strong semantic understanding, their text-based output format is inherently ill-suited for dense, precision-critical spatial predictions. Existing agentic frameworks address this limitation by delegating tasks to external tools, but indiscriminate tool invocation is computationally inefficient and underutilizes the MLLM's native capabilities. To this end, we propose RemoteAgent, an agentic framework that strategically respects the intrinsic capability boundaries of MLLMs. To empower this framework to understand real user intents, we construct VagueEO, a human-centric instruction dataset pairing EO tasks with simulated vague natural-language queries. By leveraging VagueEO for reinforcement fine-tuning, we align an MLLM into a robust cognitive core that directly resolves image- and sparse region-level tasks. Consequently, RemoteAgent processes suitable tasks internally while intelligently orchestrating specialized tools via the Model Context Protocol exclusively for dense predictions. Extensive experiments demonstrate that RemoteAgent achieves robust intent recognition capabilities while delivering highly competitive performance across diverse EO tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
