SuperCap：基于多分辨率超像素的图像描述
SuperCap: Multi-resolution Superpixel-based Image Captioning
AI总结：
本文提出SuperCap，将超像素与视觉语言模型结合并引入多分辨率输入和注意力机制，在COCO上取得136.9的CIDEr分数，摆脱了对目标检测的依赖。
摘要：
长期以来，图像描述领域的一个目标一直是摆脱对目标检测的依赖。我们研究将超像素与视觉语言模型（VLMs）相结合，以弥合基于检测器的描述架构与仅在大规模数据集上预训练的架构之间的差距。我们提出的新型超像素方法确保模型获得类似目标的特征，同时使用VLMs为模型提供开放集目标理解能力。此外，我们扩展了架构以利用多分辨率输入，使模型能够以不同细节层次查看图像，并使用注意力机制确定哪些部分与描述最相关。我们通过多个VLMs以及一系列消融实验展示了模型的性能，详细说明了不同架构选择的影响。我们的完整模型在COCO Karpathy划分上取得了具有竞争力的CIDEr分数$136.9$。
英文摘要：
It has been a longstanding goal within image captioning to move beyond a dependence on object detection. We investigate using superpixels coupled with Vision Language Models (VLMs) to bridge the gap between detector-based captioning architectures and those that solely pretrain on large datasets. Our novel superpixel approach ensures that the model receives object-like features whilst the use of VLMs provides our model with open set object understanding. Furthermore, we extend our architecture to make use of multi-resolution inputs, allowing our model to view images in different levels of detail, and use an attention mechanism to determine which parts are most relevant to the caption. We demonstrate our model's performance with multiple VLMs and through a range of ablations detailing the impact of different architectural choices. Our full model achieves a competitive CIDEr score of $136.9$ on the COCO Karpathy split.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
