CiQi-Agent:在多模态代理中对视觉、工具和美学的对齐用于中国瓷器的文化推理
CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelains
AI总结:
本文提出CiQi-Agent,通过多图像输入、视觉工具调用和多模态检索增强生成,实现对瓷器六个属性的精细分析,同时构建了专家标注的CiQi-VQA数据集和对齐的CiQi-Bench基准,实验表明其在六个属性上的准确率比GPT-5高12.2%。
摘要:
古代中国瓷器的鉴赏需要广泛的历史知识、材料理解和审美敏感性,使非专业人士难以参与。为普及文化遗产理解并协助专家鉴赏,我们引入CiQi-Agent——一个专门用于智能分析古代中国瓷器的瓷器鉴赏代理。CiQi-Agent支持多图像瓷器输入,能够调用视觉工具和多模态检索增强生成,对六个属性(朝代、 reign period、窑口、釉色、装饰图案和器形)进行精细鉴赏分析。除了属性分类,它还捕捉细微的视觉细节,检索相关领域知识,并整合视觉和文本证据,生成连贯且可解释的鉴赏描述。为此,我们构建了一个大规模、专家标注的数据集CiQi-VQA,包含29,596个瓷器样本、51,553张图像和557,940个视觉问答对,并进一步建立了与上述六个属性对齐的全面基准CiQi-Bench。CiQi-Agent通过监督微调、强化学习和工具增强推理框架进行训练,该框架整合了两种类型的工具:视觉工具和多模态检索工具。实验结果表明,CiQi-Agent(7B)在CiQi-Bench的六个属性上均优于所有竞争的开源和闭源模型,平均准确率比GPT-5高12.2%。该模型和数据集已发布并在https://huggingface.co/datasets/SII-Monument-Valley/CiQi-VQA上公开可用。
英文摘要:
The connoisseurship of antique Chinese porcelain demands extensive historical expertise, material understanding, and aesthetic sensitivity, making it difficult for non-specialists to engage. To democratize cultural-heritage understanding and assist expert connoisseurship, we introduce CiQi-Agent -- a domain-specific Porcelain Connoisseurship Agent for intelligent analysis of antique Chinese porcelain. CiQi-Agent supports multi-image porcelain inputs and enables vision tool invocation and multimodal retrieval-augmented generation, performing fine-grained connoisseurship analysis across six attributes: dynasty, reign period, kiln site, glaze color, decorative motif, and vessel shape. Beyond attribute classification, it captures subtle visual details, retrieves relevant domain knowledge, and integrates visual and textual evidence to produce coherent, explainable connoisseurship descriptions. To achieve this capability, we construct a large-scale, expert-annotated dataset CiQi-VQA, comprising 29,596 porcelain specimens, 51,553 images, and 557,940 visual question--answering pairs, and further establish a comprehensive benchmark CiQi-Bench aligned with the previously mentioned six attributes. CiQi-Agent is trained through supervised fine-tuning, reinforcement learning, and a tool-augmented reasoning framework that integrates two categories of tools: a vision tool and multimodal retrieval tools. Experimental results show that CiQi-Agent (7B) outperforms all competitive open- and closed-source models across all six attributes on CiQi-Bench, achieving on average 12.2\% higher accuracy than GPT-5. The model and dataset have been released and are publicly available at https://huggingface.co/datasets/SII-Monument-Valley/CiQi-VQA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。