大语言模型的地理空间概念探测：抽象性、组合性与接地性
Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Grounding
浏览论文内容
中文总结 AI 辅助
该研究针对LLMs的抽象性、组合性与接地性设计测试，构建空间概念基准并开展多模型实验，揭示当前LLMs的概念理解局限，为相关模型的优化提供洞见。
中文摘要 AI 辅助
理解概念是泛化能力的基础。尽管大语言模型（LLMs）在广泛任务上表现出色，但它们仍难以实现真正的概念理解。过往研究采用自然语言基准或范围狭窄的合成任务评估LLMs的概念理解，但这些设置常混淆多种技能，或缺乏对底层概念及其属性的精确控制。为支持对LLMs中概念的可控探测，我们设计了针对其核心属性——抽象性、组合性与接地性的测试。我们构建了一个以概念为中心的基准，针对方向、距离、拓扑等空间概念及其组合，并采用问答任务作为代理。我们在多种LLMs架构和训练 regime 上开展大量实验，分析模型规模与设计如何影响概念理解。结果揭示了当前LLMs的明显局限，并为理解决定其获取与组合结构化概念能力的因素提供了洞见。我们的发现为如何重新设计基于概念的LLMs以改进信息访问与知识管理提供了思路。代码将在此https URL公开。
英文摘要
Understanding concepts is fundamental to generalization. Despite their impressive performance on a wide range of tasks, Large Language Models (LLMs) still struggle with genuine concept understanding. Prior work has evaluated conceptual understanding in LLMs using natural-language benchmarks or narrowly scoped synthetic tasks, but these settings often conflate multiple skills or lack precise control over the underlying concepts and their properties. To support controlled probing of concepts in LLMs, we design tests on their core properties: abstraction, compositionality, and groundness. We set up a concept-centric benchmark, targeting spatial concepts such as direction, distance, topology, and their compositions, and use question answering tasks serving as a proxy. We conduct extensive experiments across multiple LLM architectures and training regimes to analyze how model scale and design impact conceptual understanding. The results reveal clear limitations in current LLMs and provide insights into the factors shaping their ability to acquire and compose structured concepts. Our findings shed light on how concept-based LLMs can be redesigned for improved information access and knowledge management. The code will be available at https://github.com/rd20karim/concept-probing.
发表机构
- University of Toulouse（图卢兹大学）
- IRIT（信息科学与技术研究院（IRIT））
机构由 AI 辅助整理，请以论文原文为准。
