一种评估和增强多模态大语言模型在文化情境任务中解释能力的结构框架
A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks
AI总结：
本研究提出了一种结构框架，用于评估和增强多模态大语言模型在文化情境任务中生成中国绘画批评的能力，通过量化评价特征和人设引导提示，揭示了VLMs在艺术批评领域的表现与局限。
摘要：
本研究旨在测试和评估当前主流视觉语言模型（VLMs）在生成传统中国绘画批评方面的能力和特性。为此，我们首先开发了一个定量框架用于中国绘画批评。该框架通过使用零样本分类模型从人类专家的批评中提取多维评价特征，涵盖评价立场、特征聚焦和评论质量。基于这些特征，定义并量化了几个具有代表性的批评人设。该框架随后用于评估所选VLMs，如Llama、Qwen或Gemini。实验设计涉及人设引导提示，以评估VLM生成来自不同视角的批评的能力。我们的发现揭示了VLMs在艺术批评领域当前的表现水平、优势和改进空间，为它们在复杂语义理解和内容生成任务中的潜力和局限性提供了见解。实验所用的代码可在：https://github.com/yha9806/VULCA-EMNLP2025 公开获取。
英文摘要：
This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM's ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https://github.com/yha9806/VULCA-EMNLP2025.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
