ArtRAG：面向视觉艺术理解的带结构化上下文的检索增强生成
ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding
AI总结：
针对多模态大模型难以捕捉美术作品细腻解读的问题，提出免训练框架ArtRAG，通过构建艺术上下文知识图谱并结合多粒度结构化检索引导生成，在艺术理解任务上超越多个强基线，生成更具文化内涵的解读。
摘要：
理解视觉艺术需要跨越文化、历史、风格等多个视角进行推理，而非仅靠物体识别。尽管近期的多模态大语言模型（MLLMs）在通用图像描述任务上表现出色，但它们往往难以捕捉美术作品所需的细腻解读。我们提出ArtRAG，这是一种全新的免训练框架，将结构化知识与检索增强生成（RAG）相结合，用于多视角的艺术品阐释。ArtRAG能从特定领域的文本资源中自动构建艺术上下文知识图谱（Art Context Knowledge Graph, ACKG），将艺术家、艺术运动、主题、历史事件等实体组织成丰富且可解释的图谱。推理阶段，多粒度结构化检索器会选取语义和拓扑相关的子图来引导生成，这使得MLLMs能够生成有上下文依据、兼具文化内涵的艺术描述。在SemArt和Artpedia数据集上的实验表明，ArtRAG的表现优于多个经过大量训练的基线模型。人工评估进一步证实，ArtRAG能生成连贯、有见地且富含文化底蕴的解读。
英文摘要：
Understanding visual art requires reasoning across multiple perspectives -- cultural, historical, and stylistic -- beyond mere object recognition. While recent multimodal large language models (MLLMs) perform well on general image captioning, they often fail to capture the nuanced interpretations that fine art demands. We propose ArtRAG, a novel, training-free framework that combines structured knowledge with retrieval-augmented generation (RAG) for multi-perspective artwork explanation. ArtRAG automatically constructs an Art Context Knowledge Graph (ACKG) from domain-specific textual sources, organizing entities such as artists, movements, themes, and historical events into a rich, interpretable graph. At inference time, a multi-granular structured retriever selects semantically and topologically relevant subgraphs to guide generation. This enables MLLMs to produce contextually grounded, culturally informed art descriptions. Experiments on the SemArt and Artpedia datasets show that ArtRAG outperforms several heavily trained baselines. Human evaluations further confirm that ArtRAG generates coherent, insightful, and culturally enriched interpretations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
