MultiGlobeQA：面向地理空间推理的多语言且全球多样化基准
MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning
AI总结：
MultiGlobeQA是覆盖201个国家地区的多语言地理空间推理基准，含46060个问答对，实验发现LLMs在网格索引等任务上表现差，计算是瓶颈且低收入地区表现差距大。
AI中文摘要：
地理空间推理即对现实世界实体计算距离、包含关系及其他空间关系，是导航与物流的核心，但大型语言模型（LLMs）即便存储了大量地理知识，仍难以完成所需的几何与拓扑计算。现有基准仅部分定位了这些缺陷：它们要么是合成的、规模小，要么主要是单语的，且对地理覆盖范围的控制有限。我们推出MultiGlobeQA，这是一个多语言基准，包含46060个问答对，涵盖14个空间功能家族和15种答案格式，在三个知识图谱上具备基于执行的真值。它通过按收入和密度分层抽样覆盖201个国家和地区，拥有英语及另外16种高资源与低资源语言的平行问题。在参数化、推理和智能体语境中，LLMs在需要网格索引和形状计算的任务上表现崩溃，而拓扑关系和方向任务表现最佳。检索与工具使用带来了显著提升，但即便提供了黄金事实，性能仍停留在三分之二以下，这表明瓶颈在于计算而非知识获取。模型在低收入地区的表现也不佳，而黄金事实会扩大而非缩小这一差距。
英文摘要：
Geospatial reasoning, i.e., computing distances, containment, and other spatial relations over real-world entities, is central to navigation and logistics, yet large language models (LLMs) struggle with the required geometric and topological computation despite storing considerable geographic knowledge. Existing benchmarks localize these failures only partially: they are synthetic or smallscale, largely monolingual, and offer limited control over geographic coverage. We introduce MultiGlobeQA, a multilingual benchmark of 46,060 question-answer pairs spanning 14 spatial-function families and 15 answer formats, with execution-based ground truth over three knowledge graphs. It covers 201 countries and territories via income- and density-stratified sampling, with parallel questions in English and 16 additional high- and low-resource languages. Across parametric, reasoning, and agentic settings, LLMs collapse on tasks requiring grid indexing and shape computation, while topological relations and directions fare best. Retrieval and tool use yield considerable gains, yet performance plateaus below two thirds even when gold facts are supplied, indicating that computation, not access to knowledge, is the bottleneck. Models also underperform on low-income regions, a gap that gold facts widen rather than close.
