arXiv 2602.142012026-02-20cs.CVcs.AI

GeoEyes: 面向超高清遥感影像证据驱动理解的按需视觉聚焦

GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagery

National University of Defense Technology, China（国防科技大学）；Beijing University of Posts and Telecommunications, China（北京邮电大学）；University of the Chinese Academy of Sciences, China（中国科学院大学）；Shanghai Jiao Tong University, China（上海交通大学）；Wuhan University, China（武汉大学）；Chinese Academy of Science, China（中国科学院）；Tsinghua University, China（清华大学）；Renmin University of China, China（中国人民大学）

Fengxiang Wang, Mingshuo Chen, Yueying Li, Yajie Yang, Yifan Zhang, Long Lan, Xue Yang, Hongda Sun, Yulin Wang, Di Wang, Jun Song, Jing Zhang, Bo Du

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

GeoEyes通过分阶段训练框架，结合冷启动数据集和代理强化学习方法，解决超高清遥感影像中证据获取不足的问题，提升视觉问答任务的准确性。

摘要：

基于图像的思维范式使多模态大语言模型（MLLMs）能够通过缩放工具主动探索视觉场景。这对于超高清（UHR）遥感视觉问答（VQA）至关重要，因为任务相关的线索稀疏且微小。然而，我们发现现有支持缩放的MLLMs存在一致的失败模式：工具使用同质化，即工具调用崩溃为任务无关的模式，限制了有效证据的获取。为此，我们提出了GeoEyes，一个分阶段训练框架，包括（1）一个冷启动的SFT数据集，超高清缩放链（UHR-CoZ），涵盖多样化的缩放模式，以及（2）一种代理强化学习方法AdaZoom-GRPO，该方法在缩放交互过程中明确奖励证据获取和答案改进。最终模型学会按需缩放并具备适当的停止行为，在UHR遥感基准上取得显著提升，准确率达到XLRS-Bench的54.23%。

英文摘要：

The "thinking-with-images" paradigm enables multimodal large language models (MLLMs) to actively explore visual scenes via zoom-in tools. This is essential for ultra-high-resolution (UHR) remote sensing VQA, where task-relevant cues are sparse and tiny. However, we observe a consistent failure mode in existing zoom-enabled MLLMs: Tool Usage Homogenization, where tool calls collapse into task-agnostic patterns, limiting effective evidence acquisition. To address this, we propose GeoEyes, a staged training framework consisting of (1) a cold-start SFT dataset, UHR Chain-of-Zoom (UHR-CoZ), which covers diverse zooming regimes, and (2) an agentic reinforcement learning method, AdaZoom-GRPO, that explicitly rewards evidence gain and answer improvement during zoom interactions. The resulting model learns on-demand zooming with proper stopping behavior and achieves substantial improvements on UHR remote sensing benchmarks, with 54.23% accuracy on XLRS-Bench.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。