面向零样本图像描述的大规模双向训练
Large-Scale Bidirectional Training for Zero-Shot Image Captioning
AI总结：
本文提出 BITTERS 框架，通过大规模图文双向训练实现零样本图像描述，并构建兼顾描述准确性与社会偏见的评估基准及高效关键词提取微调方法。
摘要：
在大规模数据集上训练时，图像描述模型能够理解通用领域图像的内容，但往往无法生成准确、详细的描述。为提升性能，预训练加微调一直是图像描述的关键策略。然而，我们发现图像与文本之间的大规模双向训练能够实现零样本图像描述。本文介绍了更大规模双向图像文本训练框架 BITTERS（Bidirectional Image Text Training in largER Scale），这是一个用于零样本图像描述的高效训练与推理框架。我们还提出了一个新的评估基准，由高质量数据集和一组广泛的指标组成，以恰当评估零样本描述准确性和社会偏见。此外，我们提供了一种用于关键词提取的高效微调方法。我们表明，谨慎选择大规模训练集和模型架构是实现零样本图像描述的关键。
英文摘要：
When trained on large-scale datasets, image captioning models can understand the content of images from a general domain but often fail to generate accurate, detailed captions. To improve performance, pretraining-and-finetuning has been a key strategy for image captioning. However, we find that large-scale bidirectional training between image and text enables zero-shot image captioning. In this paper, we introduce Bidirectional Image Text Training in largER Scale, BITTERS, an efficient training and inference framework for zero-shot image captioning. We also propose a new evaluation benchmark which comprises of high quality datasets and an extensive set of metrics to properly evaluate zero-shot captioning accuracy and societal bias. We additionally provide an efficient finetuning approach for keyword extraction. We show that careful selection of large-scale training set and model architecture is the key to achieving zero-shot image captioning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
