arXiv 2601.079842026-02-24cs.CL

跨文化专家级艺术批评评估与视觉-语言模型

Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models

DJCAD, University of Dundee, United Kingdom（邓迪大学DJCAD研究中心，英国）；Hebei Academy of Fine Art, China（河北美术院，中国）；Computer Science, Nanjing University, China（南京大学计算机科学系，中国）；Department of Mechanical Engineering, School of Engineering, University of Birmingham（伯明翰大学工程学院机械工程系）

Haorui Yu, Xuehang Wen, Fengrui Zhang, Qiufeng Yi

备注：: 16 pages, 7 figures, submitted to ACL 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种跨文化评估框架，用于评估视觉-语言模型在艺术批评中的文化理解能力，揭示了自动化指标与专家评分的差异及文化敏感性问题。

摘要：

视觉-语言模型（VLMs）在视觉描述方面表现出色，但在文化解释方面仍缺乏充分验证。现有基准测试仅评估感知而不涉及解释，常见的评估代理，如自动化指标和LLM-判断平均值，对于文化敏感的生成任务来说不可靠。我们通过基于艺术理论结构（第2节）的三层次评估框架来解决这一测量差距。该框架通过五个层次（L1-L5）和六个传统中的165个文化特定维度来操作文化理解。第一层次计算自动化质量指标，第二层次应用基于评分表的单判官评分，第三层次通过sigmoid校准将总分校准到人类专家评分。应用于15个VLMs的294对评估对，验证的仪器揭示了（i）自动化指标和判官评分测量不同的构念，确立单判官校准作为更可靠的替代方案；（ii）文化理解从视觉描述（L1-L2）降级到文化解释（L3-L5）；（iii）西方艺术样本比非西方样本获得更高的分数。据我们所知，这是首个跨文化评估工具用于生成艺术批评，提供了一种可重复的方法来审计VLM的文化能力。框架代码可在https://github.com/yha9806/VULCA-Framework上获得。

英文摘要：

Vision-Language Models (VLMs) excel at visual description yet remain under-validated for cultural interpretation. Existing benchmarks assess perception without interpretation, and common evaluation proxies, such as automated metrics and LLM-judge averaging, are unreliable for culturally sensitive generative tasks. We address this measurement gap with a tri-tier evaluation framework grounded in art-theoretical constructs (Section 2). The framework operationalises cultural understanding through five levels (L1--L5) and 165 culture-specific dimensions across six traditions: Tier I computes automated quality indicators, Tier II applies rubric-based single-judge scoring, and Tier III calibrates the aggregate score to human expert ratings via sigmoid calibration. Applied to 15 VLMs across 294 evaluation pairs, the validated instrument reveals that (i) automated metrics and judge scoring measure different constructs, establishing single-judge calibration as the more reliable alternative; (ii) cultural understanding degrades from visual description (L1--L2) to cultural interpretation (L3--L5); and (iii) Western art samples consistently receive higher scores than non-Western ones. To our knowledge, this is the first cross-cultural evaluation instrument for generative art critique, providing a reproducible methodology for auditing VLM cultural competence. Framework code is available at https://github.com/yha9806/VULCA-Framework.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。