ENC-Bench：用于评估多模态大语言模型在电子航海图理解中的基准
ENC-Bench: A Benchmark for Evaluating Multimodal Large Language Models in Electronic Navigational Chart Understanding
AI总结：
ENC-Bench是首个专注于专业电子航海图理解的基准，包含20490个经过专家验证的样本，评估了10种先进多模态大语言模型在符号识别、空间推理和航海决策中的表现，揭示了模型在符号 grounding、空间计算和多约束推理方面的系统性挑战。
摘要：
电子航海图（ENCs）是现代航海导航的安全关键基础，但尚不清楚多模态大语言模型（MLLMs）能否可靠解读它们。与自然图像或传统海图不同，ENCs通过标准化矢量符号、尺度依赖的渲染和精确几何结构编码法规、水深和航线约束，需要专门的航海专业知识进行解读。我们介绍了ENC-Bench，首个专注于专业ENC理解的基准。ENC-Bench包含20,490个经过专家验证的样本，来自840份真实的美国国家海洋和大气管理局（NOAA）ENCs，分为三个层级：感知（符号和特征识别）、空间推理（坐标定位、方位、距离）和航海决策（航线合法性、安全评估、多重约束下的应急规划）。所有样本均通过校准的矢量到图像管道从原始S-57数据生成，通过自动一致性检查和专家审查。我们评估了10种最先进的MLLMs，如GPT-4o、Gemini 2.5、Qwen3-VL、InternVL-3和GLM-4.5V，在统一的零样本协议下。最佳模型仅达到47.88%的准确率，存在符号 grounding、空间计算、多约束推理和对光照和尺度变化的鲁棒性方面的系统性挑战。通过建立首个严谨的ENC基准，我们开辟了专门符号推理和安全关键AI交汇的新研究前沿，为推动MLLMs向专业航海应用发展提供了必要的基础设施。
英文摘要：
Electronic Navigational Charts (ENCs) are the safety-critical backbone of modern maritime navigation, yet it remains unclear whether multimodal large language models (MLLMs) can reliably interpret them. Unlike natural images or conventional charts, ENCs encode regulations, bathymetry, and route constraints via standardized vector symbols, scale-dependent rendering, and precise geometric structure -- requiring specialized maritime expertise for interpretation. We introduce ENC-Bench, the first benchmark dedicated to professional ENC understanding. ENC-Bench contains 20,490 expert-validated samples from 840 authentic National Oceanic and Atmospheric Administration (NOAA) ENCs, organized into a three-level hierarchy: Perception (symbol and feature recognition), Spatial Reasoning (coordinate localization, bearing, distance), and Maritime Decision-Making (route legality, safety assessment, emergency planning under multiple constraints). All samples are generated from raw S-57 data through a calibrated vector-to-image pipeline with automated consistency checks and expert review. We evaluate 10 state-of-the-art MLLMs such as GPT-4o, Gemini 2.5, Qwen3-VL, InternVL-3, and GLM-4.5V, under a unified zero-shot protocol. The best model achieves only 47.88% accuracy, with systematic challenges in symbolic grounding, spatial computation, multi-constraint reasoning, and robustness to lighting and scale variations. By establishing the first rigorous ENC benchmark, we open a new research frontier at the intersection of specialized symbolic reasoning and safety-critical AI, providing essential infrastructure for advancing MLLMs toward professional maritime applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
