视觉-语言模型的文化感知能力如何？
How Culturally Aware are Vision-Language Models?
AI总结：
该研究比较了GPT-4V等四种视觉-语言模型在识别民俗图像文化信息及生成相关描述的性能，提出文化感知分数（CAS）评估指标并构建MOSAIC-1.5k数据集，以推动文化敏感型AI系统的发展。
摘要：
一幅图像通常被认为胜过千言万语，某些图像能讲述丰富而深刻的故事。这些故事能否通过图像描述来呈现？来自民俗类别的图像（如神话、民间舞蹈、文化标志和符号）对每种文化都至关重要。我们的研究比较了四种流行的视觉-语言模型（GPT-4V、Gemini Pro Vision、LLaVA和OpenFlamingo）在识别此类图像中的文化特定信息以及生成准确且具有文化敏感性的图像描述方面的性能。我们还提出了一种新的评估指标——文化感知分数（CAS），用于衡量图像描述中的文化感知程度。我们提供了一个名为MOSAIC-1.5k的数据集，其中包含具有文化背景和语境的图像的真实标签，以及一个带有文化感知分数标签的数据集，可用于未见过的数据。生成具有文化适宜性的图像描述对科学研究很有价值，也能造福许多实际应用。我们期望我们的工作能促进文化敏感性在全球人工智能应用中的更深层次整合。通过向公众提供该数据集和文化感知分数，我们旨在推动这一领域的进一步研究，鼓励开发更具文化感知能力、尊重并颂扬全球多样性的人工智能系统。
英文摘要：
An image is often considered worth a thousand words, and certain images can tell rich and insightful stories. Can these stories be told via image captioning? Images from folklore genres, such as mythology, folk dance, cultural signs, and symbols, are vital to every culture. Our research compares the performance of four popular vision-language models (GPT-4V, Gemini Pro Vision, LLaVA, and OpenFlamingo) in identifying culturally specific information in such images and creating accurate and culturally sensitive image captions. We also propose a new evaluation metric, the Cultural Awareness Score (CAS), which measures the degree of cultural awareness in image captions. We provide a dataset MOSAIC-1.5k labeled with ground truth for images containing cultural background and context and a labeled dataset with assigned Cultural Awareness Scores that can be used with unseen data. Creating culturally appropriate image captions is valuable for scientific research and can be beneficial for many practical applications. We envision our work will promote a deeper integration of cultural sensitivity in AI applications worldwide. By making the dataset and Cultural Awareness Score available to the public, we aim to facilitate further research in this area, encouraging the development of more culturally aware AI systems that respect and celebrate global diversity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
