用于在无额外数据下提升图像描述能力的通信博弈涌现自然语言
Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data
AI总结：
针对无监督图像描述性能不足的问题，提出多智能体强化学习博弈LoGIC，通过GRPO算法训练说话者与倾听者协作交流，在无额外标签下显著提升了图像描述性能。
摘要：
图像描述是开发各类AI系统的重要问题，这些任务需要大量标注图像来训练模型。由于所有现有标注数据集均已用于训练大型视觉语言模型（VLMs），提升其性能变得具有挑战性。鉴于此，必须考虑相对缺乏探索的无监督图像描述性能。为此，我们提出LoGIC（Lewis Communication Game for Image Captioning），一种多智能体强化学习博弈。该方法包含两个智能体：“说话者”和“倾听者”，目标是学习用自然语言交流的策略。我们使用GRPO算法在合作共同奖励设置下训练智能体，并表明图像描述性能的提升是智能体学习玩游戏的结果。我们展示，使用预训练VLMs作为“说话者”和大型语言模型（LLM）作为“倾听者”进行语言理解，在通过LoGIC微调且无额外标签后，我们获得了46的BLEU分数，相比原始VLM的44 BLEU分数，在绝对指标上有2个单位的优势。此外，我们用轻量级组件替换“说话者”中的VLM：(i)用于图像感知的ViT和(ii)用于语言生成的GPT2，并使用LoGIC从头训练它们，在无监督设置下获得31的BLEU分数，比现有无监督图像描述方法有10分的优势。
英文摘要：
Image captioning is an important problem in developing various AI systems, and these tasks require large volumes of annotated images to train the models. Since all existing labelled datasets are already used for training the large Vision Language Models (VLMs), it becomes challenging to improve the performance of the same. Considering this, it is essential to consider the unsupervised image captioning performance, which remains relatively under-explored. To that end, we propose LoGIC (Lewis Communication Game for Image Captioning), a Multi-agent Reinforcement Learning game. The proposed method consists of two agents, a 'speaker' and a 'listener', with the objective of learning a strategy for communicating in natural language. We train agents in the cooperative common-reward setting using the GRPO algorithm and show that improvement in image captioning performance emerges as a consequence of the agents learning to play the game. We show that using pre-trained VLMs as the 'speaker' and Large Language Model (LLM) for language understanding in the 'listener', we achieved a $46$ BLEU score after fine-tuning using LoGIC without additional labels, a $2$ units advantage in absolute metrics compared to the $44$ BLEU score of the vanilla VLM. Additionally, we replace the VLM from the 'speaker' with lightweight components: (i) a ViT for image perception and (ii) a GPT2 language generation, and train them from scratch using LoGIC, obtaining a $31$ BLEU score in the unsupervised setting, a $10$ points advantage over existing unsupervised image-captioning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
