当模型误读墨迹：审计视觉语言模型（VLM）的阿拉伯书法识别能力
When Models Misread Ink: Auditing VLMs for Arabic Calligraphy Recognition
AI总结：
该研究审计12个VLM在阿拉伯书法识别任务上的表现，发现存在默认标签崩溃问题，阿拉伯书法识别可作为测试VLM鲁棒性的文化压力工具。
摘要：
阿拉伯书法是穆斯林社区中高风险的文化视觉领域，但视觉语言模型（VLM）的评估很少测试其精细的书法传统。我们使用数字提示、视觉线索提示和优先迪瓦尼（Diwani）提示，在包含1195张图像的六分类清理基准上审计了12个VLM。在这个闭集设置中，最佳数字提示准确率对于闭集/API模型为60.9%，对于开放权重模型为47.2%。主要失败是默认标签崩溃：每次数字运行都被单个标签主导，占预测的33-63%，而多数类基线为18.8%。12个模型中有8个的默认标签是苏鲁斯体（Thuluth）；GPT-4o将52.9%的所有图像标记为苏鲁斯体，尽管苏鲁斯体仅占数据集的18.1%。提示的帮助不均衡：视觉线索使一些模型的准确率提高了10个以上百分点，但对另一些模型则有损害；突出迪瓦尼（Diwani）通常只是转移了默认标签，而不是消除它。因此，阿拉伯书法识别是一种紧凑的、基于文化的VLM鲁棒性压力测试，可用于衡量哪些视觉传统对自动化系统变得可识别。
英文摘要：
Arabic calligraphy is a high-stakes cultural visual domain for Muslim communities, yet VLM evaluations rarely test its fine-grained script traditions. We audit 12 VLMs on a cleaned 1,195-image six-way benchmark using numeric, visual-cue, and Diwani-first prompts. In this closed-set setting, the best numeric-prompt accuracy is 60.9% for closed/API models and 47.2% for open-weight models. The main failure is default-label collapse: every numeric run is dominated by a single label taking 33-63% of predictions, against an 18.8% majority-class baseline. For 8 of 12 models the default is Thuluth; GPT-4o labels 52.9% of all images as Thuluth although it is only 18.1% of the set. Prompting helps unevenly: visual cues move accuracy by more than 10 points for some models but hurt others, and foregrounding Diwani often relocates the default rather than removing it. Arabic calligraphy recognition is therefore a compact, culturally grounded stress test for VLM robustness and for measuring which visual traditions become legible to automated systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
