基于自训练奖励模型的流畅且准确的图像描述生成
Fluent and Accurate Image Captioning with a Self-Trained Reward Model
AI总结：
本文提出Self-Cap方法，通过基于自生成负样本训练的可学习判别器作为奖励模型，在提升图像描述正确性与丰富度的同时避免CLIP奖励导致的语法问题，并缩短了微调时间。
摘要：
使用CIDEr等手工设计的奖励指标对图像描述模型进行微调，一直是提升序列级描述质量的经典策略。然而，这种方法已知会限制描述的丰富性和语义深度，并倾向于使模型偏向于真实标注句子的风格，从而丢失细节和特异性。相反，近期尝试使用CLIP等图像-文本模型作为奖励，却导致了语法错误和重复的描述。在本文中，我们提出了Self-Cap，这是一种图像描述方法，它依赖于一个基于自生成负样本的可学习奖励模型，该模型能够根据描述与图像的一致性来区分描述。具体而言，我们的判别器是一个经过微调的对比式图像-文本模型，训练目标是提升描述的正确性，同时避免在使用基于CLIP的奖励进行训练时通常出现的异常现象。为此，我们的判别器直接引入了来自冻结描述器的负样本，这不仅显著提高了生成描述的质量和丰富度，而且与仅使用CIDEr分数作为优化指标相比，还减少了微调时间。实验结果证明了我们的训练策略在标准图像描述数据集和零样本图像描述数据集上的有效性。
英文摘要：
Fine-tuning image captioning models with hand-crafted rewards like the CIDEr metric has been a classical strategy for promoting caption quality at the sequence level. This approach, however, is known to limit descriptiveness and semantic richness and tends to drive the model towards the style of ground-truth sentences, thus losing detail and specificity. On the contrary, recent attempts to employ image-text models like CLIP as reward have led to grammatically incorrect and repetitive captions. In this paper, we propose Self-Cap, a captioning approach that relies on a learnable reward model based on self-generated negatives that can discriminate captions based on their consistency with the image. Specifically, our discriminator is a fine-tuned contrastive image-text model trained to promote caption correctness while avoiding the aberrations that typically happen when training with a CLIP-based reward. To this end, our discriminator directly incorporates negative samples from a frozen captioner, which significantly improves the quality and richness of the generated captions but also reduces the fine-tuning time in comparison to using the CIDEr score as the sole metric for optimization. Experimental results demonstrate the effectiveness of our training strategy on both standard and zero-shot image captioning datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
