arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.22148cs.CV
收藏

OpenEarth-Agent:从工具调用到工具创建用于开放环境地球观测

OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation

Nanjing University(南京大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);The University of Tokyo(东京大学)

Sijie Zhao, Feng Liu, Xueliang Zhang, Hao Chen, Xinyu Gu, Zhe Jiang, Fenghua Ling, Ben Fei, Wenlong Zhang, Junjue Wang, Weihao Xuan, Pengfeng Xiao, Naoto Yokoya, Lei Bai

AI总结:

本文提出OpenEarth-Agent框架,通过自适应工作流规划与工具创建,解决开放环境地球观测中多源数据和异构任务的挑战,并通过OpenEarth-Bench评估其适应性和工具创建能力。

摘要:

地球观测(EO)对于感知动态地表变化至关重要,但在开放环境中部署自主EO受到多源数据和异构任务的极大多样性阻碍。尽管远程传感代理已出现以简化EO工作流程,但现有工具调用代理局限于封闭环境。它们依赖于预定义工具且范围狭窄,限制了其在多样化数据和任务中的泛化能力。为克服这些限制,我们引入了OpenEarth-Agent,这是首个针对开放环境EO的工具创建代理框架。不同于调用预定义工具,OpenEarth-Agent采用自适应工作流规划和工具创建以泛化到未见过的数据和任务。这种适应性通过多阶段工具和跨领域知识库的开放性整合得到加强,使在多个应用领域中整个EO流程的稳健执行成为可能。为全面评估开放环境中的EO代理,我们提出OpenEarth-Bench,包含596个真实世界、全流程案例,涵盖七个应用领域,专门评估代理的适应性规划和工具创建能力。该基准仅提供必要的预训练模型工具,不包含其他预定义任务特定工具。大量实验表明,OpenEarth-Agent成功掌握了多个领域的全流程EO。值得注意的是,在跨基准Earth-Bench中,配备6个必要预训练模型的工具创建代理在性能上与依赖104个专用工具的工具调用代理相当,并在提供完整工具集时显著优于它们。在某些情况下,创建的工具比人工设计的工具在数据异常面前更具鲁棒性。

英文摘要:

Earth Observation (EO) is essential for perceiving dynamic land surface changes, yet deploying autonomous EO in open environments is hindered by the immense diversity of multi-source data and heterogeneous tasks. While remote sensing agents have emerged to streamline EO workflows, existing tool-calling agents are confined to closed environments. They rely on pre-defined tools and are restricted to narrow scope, limiting their generalization to the diverse data and tasks. To overcome these limitations, we introduce OpenEarth-Agent, the first tool-creation agent framework tailored for open-environment EO. Rather than calling predefined tools, OpenEarth-Agent employs adaptive workflow planning and tool creation to generalize to unseen data and tasks. This adaptability is bolstered by an open-ended integration of multi-stage tools and cross-domain knowledge bases, enabling robust execution in the entire EO pipeline across multiple application domains. To comprehensively evaluate EO agents in open environments, we propose OpenEarth-Bench, a novel benchmark comprising 596 real-world, full-pipeline cases across seven application domains, explicitly designed to assess agents' adaptive planning and tool creation capabilities. Only essential pre-trained model tools are provided in this benchmark, devoid of any other predefined task-specific tools. Extensive experiments demonstrate that OpenEarth-Agent successfully masters full-pipeline EO across multiple domains in the open environment. Notably, on the cross-benchmark Earth-Bench, our tool-creating agent equipped with 6 essential pre-trained models achieves performance comparable to tool-calling agents relying on 104 specialized tools, and significantly outperforms them when provided with the complete toolset. In several cases, the created tools exhibit superior robustness to data anomalies compared to human-engineered counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑