打造中国文化之镜：一个用于中国谐音双关艺术理解的多模态数据集
Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding
AI总结：
本文构建深植中国传统文化的多模态 Pun Rebus Art Dataset，通过元素识别、象征匹配和信息解释任务评估 VLM，揭示其文化理解短板并推动更具包容性的模型发展。
摘要：
大型视觉-语言模型（VLM）在理解日常内容方面已展现出卓越能力。然而，它们在艺术领域，尤其是文化内涵丰富的艺术形式中的表现仍较少被探索。作为人类智慧与创造力的明珠，艺术蕴含着复杂的文化叙事与象征意义。本文提出 Pun Rebus Art Dataset，这是一个深植于中国传统文化、用于艺术理解的多模态数据集。我们聚焦三项主要任务：识别显著视觉元素、将元素与其象征意义进行匹配，以及解释所传达的信息。评估表明，最先进的 VLM 在这些任务上表现吃力，往往给出带有偏见且产生幻觉的解释，并且通过上下文学习仅能获得有限提升。通过发布 Pun Rebus Art Dataset，我们旨在推动能够更好理解和诠释特定文化内容的 VLM 发展，促进超越基于英语语料库的更强包容性。
英文摘要：
Large vision-language models (VLMs) have demonstrated remarkable abilities in understanding everyday content. However, their performance in the domain of art, particularly culturally rich art forms, remains less explored. As a pearl of human wisdom and creativity, art encapsulates complex cultural narratives and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal dataset for art understanding deeply rooted in traditional Chinese culture. We focus on three primary tasks: identifying salient visual elements, matching elements with their symbolic meanings, and explanations for the conveyed messages. Our evaluation reveals that state-of-the-art VLMs struggle with these tasks, often providing biased and hallucinated explanations and showing limited improvement through in-context learning. By releasing the Pun Rebus Art Dataset, we aim to facilitate the development of VLMs that can better understand and interpret culturally specific content, promoting greater inclusiveness beyond English-based corpora.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
