CIC：一个文化感知图像描述生成框架
CIC: A Framework for Culturally-Aware Image Captioning
AI总结：
针对现有图像描述方法缺乏对文化元素详细描述的问题，提出文化感知图像描述框架CIC，通过生成文化问题、VQA提取视觉元素及LLM生成描述，人工评估表明其描述更具文化性。
摘要：
图像描述生成利用视觉-语言预训练模型（如BLIP）从图像生成描述性句子，该领域已取得显著进步。然而，现有方法缺乏对图像中描绘的文化元素（如亚洲文化群体中人们穿着的传统服饰）生成详细描述性说明的能力。在本文中，我们提出一个新框架——文化感知图像描述生成（CIC），该框架生成描述并阐述从代表不同文化的图像中的文化视觉元素所提取的文化元素。受将视觉模态与大型语言模型（LLMs）通过适当提示相结合的方法启发，我们的框架（1）基于图像中的文化类别生成问题，（2）利用生成的问题通过视觉问答（VQA）提取文化视觉元素，以及（3）使用带有提示的LLMs生成文化感知的描述。我们对来自4个不同文化群体、对相应文化有高度理解的45名参与者进行的人工评估表明，与基于视觉-语言预训练模型的图像描述基线相比，我们提出的框架生成的描述更具文化描述性。资源可在https://shane3606.github.io/cic获取。
英文摘要：
Image Captioning generates descriptive sentences from images using Vision-Language Pre-trained models (VLPs) such as BLIP, which has improved greatly. However, current methods lack the generation of detailed descriptive captions for the cultural elements depicted in the images, such as the traditional clothing worn by people from Asian cultural groups. In this paper, we propose a new framework, Culturally-aware Image Captioning (CIC), that generates captions and describes cultural elements extracted from cultural visual elements in images representing cultures. Inspired by methods combining visual modality and Large Language Models (LLMs) through appropriate prompts, our framework (1) generates questions based on cultural categories from images, (2) extracts cultural visual elements from Visual Question Answering (VQA) using generated questions, and (3) generates culturally-aware captions using LLMs with the prompts. Our human evaluation conducted on 45 participants from 4 different cultural groups with a high understanding of the corresponding culture shows that our proposed framework generates more culturally descriptive captions when compared to the image captioning baseline based on VLPs. Resources can be found at https://shane3606.github.io/cic..
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
