多模态大语言模型在离散符号理解中的认知不匹配
Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding
AI总结：
研究发现多模态大语言模型在离散符号识别上表现欠佳，而在复杂推理任务上表现较好，揭示了当前系统依赖语言先验而非稳健视觉基础的问题。
摘要：
多模态大语言模型（MLLMs）在自然图像上表现优异，但对其离散视觉符号的理解能力仍不明确。本文提出一个跨领域基准，涵盖语言、文化、数学、物理和化学，分为三个认知层级：感知与识别、组合与推理、关联与批判性思维。在领先MLLMs上观察到一致的认知不匹配：模型在基本符号识别上表现不佳，而在更复杂的推理任务上表现较好。这种识别-推理倒置表明，当前系统常依赖语言先验、模板检索或程序推理，而非稳健的视觉基础。该模式在稀疏、低冗余符号（如手写字符、公式图、电路图和化学结构）中尤为明显。这些结果表明，符号理解仍是多模态智能的主要瓶颈，激励了优先考虑离散语义空间中基础感知的训练和评估方案。
英文摘要：
Multimodal large language models (MLLMs) perform strongly on natural images, yet their ability to understand discrete visual symbols remains unclear. We present a multi-domain benchmark spanning language, culture, mathematics, physics and chemistry, organized into three cognitive levels: perception and recognition, combination and reasoning, and association and critical thinking. Across leading MLLMs, we observe a consistent cognitive mismatch. Models frequently underperform on elementary symbol recognition while appearing relatively competent on more complex reasoning tasks. This recognition-reasoning inversion indicates that current systems often compensate with linguistic priors, template retrieval or procedural reasoning instead of robust visual grounding. The pattern is especially clear for sparse, low-redundancy symbols such as handwritten characters, formula graphs, circuit diagrams and chemical structures. These results show that symbolic understanding remains a major bottleneck for multimodal intelligence and motivate training and evaluation schemes that prioritize grounded perception in discrete semantic spaces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
