CVPR2024 NICE图像描述挑战赛的解决方案
The Solution for the CVPR2024 NICE Image Captioning Challenge
AI总结：
本报告提出了一种针对CVPR2024 NICE零样本图像描述挑战的解决方案，通过检索增强和描述分级方法，结合OFA模型与高质量训练数据，实现了234.11的CIDEr得分。
摘要：
本报告介绍了针对2024 NICE：零样本图像描述评估新前沿的Topic 1零样本图像描述的解决方案。与NICE 2023数据集相比，该挑战包含由人类进行的新标注，在描述风格和内容上存在显著差异。因此，我们通过检索增强和描述分级方法有效提升了图像描述质量。在数据层面，我们利用图像描述模型生成的高质量描述作为训练数据，以弥补文本风格的差异。在模型层面，我们采用OFA（一种基于手工模板的大规模视觉-语言预训练模型）来执行图像描述任务。随后，我们针对图像描述模型生成的高质量描述数据提出描述级策略，并将其与检索增强策略整合到模板中，促使模型基于检索增强提示生成更高质量、更匹配且语义更丰富的描述。我们的方法实现了234.11的CIDEr得分。
英文摘要：
This report introduces a solution to the Topic 1 Zero-shot Image Captioning of 2024 NICE : New frontiers for zero-shot Image Captioning Evaluation. In contrast to NICE 2023 datasets, this challenge involves new annotations by humans with significant differences in caption style and content. Therefore, we enhance image captions effectively through retrieval augmentation and caption grading methods. At the data level, we utilize high-quality captions generated by image caption models as training data to address the gap in text styles. At the model level, we employ OFA (a large-scale visual-language pre-training model based on handcrafted templates) to perform the image captioning task. Subsequently, we propose caption-level strategy for the high-quality caption data generated by the image caption models and integrate them with retrieval augmentation strategy into the template to compel the model to generate higher quality, more matching, and semantically enriched captions based on the retrieval augmentation prompts. Our approach achieves a CIDEr score of 234.11.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
