arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.18792cs.CYcs.CL
收藏

用于自动化地理空间数据分析的 LLM 智能体

An LLM Agent for Automatic Geospatial Data Analysis

Yuxing Chen, Weijie Wang, Sylvain Lobry, Camille Kurtz

AI总结:

本文提出 GeoAgent 框架,在 MCTS 中融合代码解释器、静态分析与 RAG,并构建地理空间任务基准,显著提升 LLM 的函数调用与任务完成能力。

摘要:

大语言模型(LLM)正被用于数据科学代码生成任务,但它们在处理复杂的顺序任务时往往遇到困难,从而导致逻辑错误。由于难以纳入复杂的数据结构和空间约束、难以有效利用多样化的函数调用,并且倾向于幻觉出较少使用的地理空间库,因此将其应用于地理空间数据处理尤其具有挑战性。为解决这些问题,我们提出 GeoAgent,这是一种新的交互式框架,旨在帮助 LLM 更有效地处理地理空间数据。GeoAgent 率先在 Monte Carlo Tree Search(MCTS,蒙特卡洛树搜索)算法中集成代码解释器、静态分析和 Retrieval-Augmented Generation(RAG,检索增强生成)技术,为地理空间数据处理提供了一种新颖方法。此外,我们还贡献了一个专门用于评估基于 LLM 的地理空间任务方法的新基准。该基准利用多种 Python 库,包含数据获取、数据分析和可视化等单轮与多轮任务。通过在多样化地理空间语境中提供全面评估,该基准为开发面向地理空间数据分析任务的基于 LLM 的方法设立了新标准。我们的研究结果表明,仅依赖 LLM 的知识不足以准确完成地理空间任务编程,因为这类任务需要连贯的多步骤流程和多次函数调用。与基线 LLM 相比,所提出的 GeoAgent 展现出更优性能,在函数调用和任务完成方面取得显著提升。此外,这些结果为未来在自动化地理空间数据分析任务编程中开发 LLM 智能体提供了宝贵见解。

英文摘要:

Large language models (LLMs) are being used in data science code generation tasks, but they often struggle with complex sequential tasks, leading to logical errors. Their application to geospatial data processing is particularly challenging due to difficulties in incorporating complex data structures and spatial constraints, effectively utilizing diverse function calls, and the tendency to hallucinate less-used geospatial libraries. To tackle these problems, we introduce GeoAgent, a new interactive framework designed to help LLMs handle geospatial data processing more effectively. GeoAgent pioneers the integration of a code interpreter, static analysis, and Retrieval-Augmented Generation (RAG) techniques within a Monte Carlo Tree Search (MCTS) algorithm, offering a novel approach to geospatial data processing. In addition, we contribute a new benchmark specifically designed to evaluate the LLM-based approach in geospatial tasks. This benchmark leverages a variety of Python libraries and includes both single-turn and multi-turn tasks such as data acquisition, data analysis, and visualization. By offering a comprehensive evaluation among diverse geospatial contexts, this benchmark sets a new standard for developing LLM-based approaches in geospatial data analysis tasks. Our findings suggest that relying solely on knowledge of LLM is insufficient for accurate geospatial task programming, which requires coherent multi-step processes and multiple function calls. Compared to the baseline LLMs, the proposed GeoAgent has demonstrated superior performance, yielding notable improvements in function calls and task completion. In addition, these results offer valuable insights for the future development of LLM agents in automatic geospatial data analysis task programming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑