TROPE：无训练对象部件增强，无缝提升细粒度零样本图像描述生成
TROPE: TRaining-Free Object-Part Enhancement for Seamlessly Improving Fine-Grained Zero-Shot Image Captioning
AI总结：
针对细粒度零样本图像描述生成性能不足的问题，提出无训练对象部件增强（TROPE），利用检测器与NLP补充部件细节，无缝提升各方法性能并达到最先进水平。
摘要：
零样本推理，即预训练模型在没有特定训练数据的情况下执行任务，是像CLIP这样的大模型令人兴奋的新兴能力。尽管在提升图像描述生成（IC）在MSCOCO和Flickr8k等流行数据集上的零样本能力方面已有大量探索，但这些方法在CUB、FLO、UCM-Captions和Sydney-Captions等细粒度数据集上表现不足。这些数据集要求描述能够区分视觉和语义上相似的类别，重点关注详细的物体部件及其属性。为克服这一挑战，我们引入了无训练对象部件增强（TROPE）。TROPE利用目标检测器提议和自然语言处理技术，为基础描述补充额外的对象部件细节。它是对基础描述的补充而非修改，从而能够与其他描述生成方法无缝集成，并为用户提供更高的灵活性。我们的评估表明，TROPE在所有测试的零样本IC方法上持续提升性能，并在细粒度IC数据集上取得了最先进的结果。
英文摘要：
Zero-shot inference, where pre-trained models perform tasks without specific training data, is an exciting emergent ability of large models like CLIP. Although there has been considerable exploration into enhancing zero-shot abilities in image captioning (IC) for popular datasets such as MSCOCO and Flickr8k, these approaches fall short with fine-grained datasets like CUB, FLO, UCM-Captions, and Sydney-Captions. These datasets require captions to discern between visually and semantically similar classes, focusing on detailed object parts and their attributes. To overcome this challenge, we introduce TRaining-Free Object-Part Enhancement (TROPE). TROPE enriches a base caption with additional object-part details using object detector proposals and Natural Language Processing techniques. It complements rather than alters the base caption, allowing seamless integration with other captioning methods and offering users enhanced flexibility. Our evaluations show that TROPE consistently boosts performance across all tested zero-shot IC approaches and achieves state-of-the-art results on fine-grained IC datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
