图像字幕生成的纯文本合成
Text-only Synthesis for Image Captioning
AI总结：
提出纯文本合成方法ToCa，通过解构字幕为结构与词汇词并组合输入大语言模型生成新字幕，实现跨领域、领域内及数据高效三种场景下的图像字幕生成，零样本CIDEr提升近5分，数据高效场景提升超20分。
摘要：
从配对图像-文本训练到纯文本训练的图像字幕生成，追求放宽对高质量数据的高成本和大规模标注要求的目标始终如一。本文提出纯文本合成用于图像字幕生成（ToCa），该方法以更少的人力和计算时间进一步推进了这一放宽。具体而言，我们将字幕文本解构为结构和词汇词，它们作为字幕的基本组成部分。通过将不同的结构和词汇词组合作为大型语言模型的输入，生成了包含各种词汇词模式的大量字幕。该方法不仅接近目标领域，而且通过生成新字幕超越了目标领域，从而增强了模型的零样本泛化能力。考虑到现实世界中数据访问的不同级别，我们定义了三种合成场景：跨领域合成、领域内合成和数据高效合成。这些场景中的实验证明了ToCa的泛化性、迁移性和实用性，零样本跨领域字幕生成的CIDEr得分提高了近5分，数据高效字幕生成的最大提升超过20 CIDEr。
英文摘要：
From paired image-text training to text-only training for image captioning, the pursuit of relaxing the requirements for high-cost and large-scale annotation of good quality data remains consistent. In this paper, we propose Text-only Synthesis for Image Captioning (ToCa), which further advances this relaxation with fewer human labor and less computing time. Specifically, we deconstruct caption text into structures and lexical words, which serve as the fundamental components of the caption. By combining different structures and lexical words as inputs to the large language model, massive captions that contain various patterns of lexical words are generated. This method not only approaches the target domain but also surpasses it by generating new captions, thereby enhancing the zero-shot generalization ability of the model. Considering the different levels of data access in the real world, we define three synthesis scenarios: cross-domain synthesis, in-domain synthesis, and data-efficient synthesis. Experiments in these scenarios demonstrate the generalizability, transferability and practicability of ToCa with a nearly 5 CIDEr improvement for zero-shot cross-domain captioning and a maximum increase of over 20 CIDEr for data-efficient captioning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
