众人的力量：用于文化图像描述的多智能体多模态模型
The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning
AI总结：
本文提出多智能体框架MosAIC，通过赋予不同文化角色提升大型多模态模型的跨文化图像描述能力，并构建多文化数据集与可适应文化的评估指标，证明多智能体交互优于单智能体模型。
摘要：
大型多模态模型（LMMs）在各种多模态任务中展现出令人印象深刻的性能。然而，由于大多数数据和模型主要以西方为中心，它们在跨文化语境中的有效性仍然有限。相反，多智能体模型在解决复杂任务方面已显示出显著的能力。我们的研究评估了LMMs在多智能体交互环境中针对文化图像描述这一新任务的集体性能。我们的贡献如下：（1）我们引入了MosAIC，这是一个使用具有不同文化角色的LMMs来增强跨文化图像描述的多智能体框架；（2）我们提供了一个包含来自中国、印度和罗马尼亚图像的丰富文化信息的英文图像描述数据集，涵盖三个数据集：GeoDE、GD-VCR、CVQA；（3）我们提出了一种可适应文化的指标，用于评估图像描述中的文化信息；（4）我们表明多智能体交互在不同指标上优于单智能体模型，并为未来研究提供了有价值的见解。我们的数据集和模型可在 https://github.com/MichiganNLP/MosAIC 获取。
英文摘要：
Large Multimodal Models (LMMs) exhibit impressive performance across various multimodal tasks. However, their effectiveness in cross-cultural contexts remains limited due to the predominantly Western-centric nature of most data and models. Conversely, multi-agent models have shown significant capability in solving complex tasks. Our study evaluates the collective performance of LMMs in a multi-agent interaction setting for the novel task of cultural image captioning. Our contributions are as follows: (1) We introduce MosAIC, a Multi-Agent framework to enhance cross-cultural Image Captioning using LMMs with distinct cultural personas; (2) We provide a dataset of culturally enriched image captions in English for images from China, India, and Romania across three datasets: GeoDE, GD-VCR, CVQA; (3) We propose a culture-adaptable metric for evaluating cultural information within image captions; and (4) We show that the multi-agent interaction outperforms single-agent models across different metrics, and offer valuable insights for future research. Our dataset and models can be accessed at https://github.com/MichiganNLP/MosAIC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
