大型视觉-语言模型是否已掌握艺术史？
Have Large Vision-Language Models Mastered Art History?
AI总结：
本文首次系统评估CLIP、LLaVA和GPT-4o三种大型视觉-语言模型在绘画风格、作者和时期零样本分类上的能力，并与人类艺术史专家对比，揭示其推理与分类模式。
摘要：
大型视觉-语言模型（VLMs）的出现，已在多个领域的图像分类中确立了新的基线。我们考察其多模态推理能力是否也能应对人类专家所掌握的挑战。具体而言，我们测试VLM能否对绘画的风格、作者和创作日期进行分类，这一领域传统上由艺术史学家所精通。与自然图像相比，艺术品因其固有复杂且多样的结构（以可变的构图和风格为特征）而构成独特挑战。这需要情境化和风格化的解读，而非简单的物体识别。艺术史学家长期研究艺术品的独特方面，其中风格预测是其学科的关键组成部分。本文探讨了集成视觉和文本数据的大型VLM能否有效推理绘画的历史和风格属性。我们首次开展了此类研究，对三种VLM（即CLIP、LLaVA和GPT-4o）进行了深入分析，评估其在艺术风格、作者和时期上的零样本分类能力。利用两个艺术品图像基准，我们评估了模型解读风格的能力、对提示的敏感性，并检查了失败案例。此外，我们通过分析错误分类，重点关注这些模型与人类艺术史专业知识相比的表现，从而深入理解其推理和分类模式。
英文摘要：
The emergence of large Vision-Language Models (VLMs) has established new baselines in image classification across multiple domains. We examine whether their multimodal reasoning can also address a challenge mastered by human experts. Specifically, we test whether VLMs can classify the style, author and creation date of paintings, a domain traditionally mastered by art historians. Artworks pose a unique challenge compared to natural images due to their inherently complex and diverse structures, characterized by variable compositions and styles. This requires a contextual and stylistic interpretation rather than straightforward object recognition. Art historians have long studied the unique aspects of artworks, with style prediction being a crucial component of their discipline. This paper investigates whether large VLMs, which integrate visual and textual data, can effectively reason about the historical and stylistic attributes of paintings. We present the first study of its kind, conducting an in-depth analysis of three VLMs, namely CLIP, LLaVA, and GPT-4o, evaluating their zero-shot classification of art style, author and time period. Using two image benchmarks of artworks, we assess the models' ability to interpret style, evaluate their sensitivity to prompts, and examine failure cases. Additionally, we focus on how these models compare to human art historical expertise by analyzing misclassifications, providing insights into their reasoning and classification patterns.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
