arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.22367cs.CLcs.AI
收藏

CNSL-bench：用于评估多模态大语言模型在中文国家手语理解能力的基准

CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language

School of Informatics, Xiamen University, China（厦门大学信息学院）；Key Lab of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian-Taiwan (XMU), Ministry of Culture and Tourism, China（福建省-台湾非物质文化遗产数字化保护与智能处理重点实验室（XMU），文化和旅游部，中国）；National Language Resources Monitoring and Research Center for Education and Teaching Media, Xiamen University, China（教育与教学媒体语言资源监测与研究中心，厦门大学，中国）

Rui Zhao, Xuewen Zhong, Xiaoyun Zheng, Jinsong Su, Yidong Chen

备注：
Accepted as the Main Conference at ACL 2026
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出CNSL-bench，首个评估多模态大语言模型在中文国家手语理解能力的基准，通过权威 grounding、多模态覆盖和手部动作多样性，评估21个模型，发现当前模型在手语理解上仍显著劣于人类表现。

摘要：

手语研究因大语言模型（LLMs）的进步而取得显著进展，但LLMs理解手语，尤其是多模态情境中的能力仍待探索。为此，我们引入CNSL-bench，首个针对评估多模态大语言模型（MLLMs）在手语理解能力的中文国家手语基准。所提CNSL-bench具有三个特点：1）权威 grounding，基于官方标准化的《国家通用手语词典》，减少区域或非标准变体的歧义，确保一致的语义定义；2）多模态覆盖，提供对齐的文本描述、示意图像和手语视频；3）手部动作多样性，支持在关键手部动作形式上进行细粒度分析，包括书写、指法和中文手语字母。利用CNSL-bench，我们广泛评估了21个开源和专有最新MLLMs。结果表明，尽管在多模态建模方面有进展，当前MLLMs在手语理解上仍显著劣于人类表现，表现出输入模态和手部动作形式之间的系统性差异。进一步的诊断分析表明，性能限制不仅存在于推理改进之外，而且指令遵循的鲁棒性在不同模型之间存在显著差异。

英文摘要：

Sign language research has achieved significant progress due to the advances in large language models (LLMs). However, the intrinsic ability of LLMs to understand sign language, especially in multimodal contexts, remains underexplored. To address this limitation, we introduce CNSL-bench, the first comprehensive Chinese em{National Sign Language benchmark designed for evaluating multimodal large language models (MLLMs) in sign language understanding. The proposed CNSL-bench is characterized by: 1) Authoritative grounding, as it is anchored to the officially standardized \textit{National Common Sign Language Dictionary, mitigating ambiguity from regional or non-canonical variants and ensuring consistent semantic definitions; 2) Multimodal coverage, providing aligned textual descriptions, illustrative images, and sign language videos; and 3) Articulatory diversity, supporting fine-grained analysis across key manual articulatory forms, including air-writing, finger-spelling, and the Chinese manual-alphabet. Using CNSL-bench, we extensively evaluate 21 open-source and proprietary up-to-date MLLMs. Our results reveal that, despite recent advances in multimodal modeling, current MLLMs remain substantially inferior to human performance, exhibiting systematic disparities across input modalities and manual articulatory forms. Additional diagnostic analyses suggest that several performance limitations persist beyond improvements in reasoning and that instruction-following robustness varies substantially across models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑