面向多样化字幕生成的图像嵌入采样方法
Image Embedding Sampling Method for Diverse Captioning
AI总结：
针对小型视觉语言模型字幕细节不足、大型模型计算成本高的问题，提出一种无需训练的图像嵌入采样框架，以BLIP为骨干结合结构化分割的层级表征，在多个数据集上实现了媲美大模型的多样性与语义性能。
摘要：
当前最先进的视觉语言模型（VLM）在图像字幕生成任务上的表现随时间推移大幅提升，但这是以计算复杂度升高为代价的，导致它们难以应用于移动设备、辅助技术等资源受限场景。相比之下，规模较小的VLM更侧重高层场景描述，却忽略了有助于更丰富图像理解所需的细节。本文提出一种无需训练的框架，以规模较小的VLM——BLIP作为骨干网络，通过显式关注不同的图像区域来提升字幕的多样性与信息量。我们的方法利用结构化分割生成层级表征，同时捕捉全局与局部语义。无需额外模型训练，实验证明该方法能让小型VLM在图文对齐度、语义完整性和多样性上达到与大型模型相当的性能。我们在MSCOCO、Flickr30k和Nocaps测试数据集上评估了该框架，三个数据集的Div-2得分分别为0.735、0.750和0.748，同时保持了与人工标注字幕相当的强图文相关性和语义完整性。
英文摘要：
Image Captioning for state-of-the-art VLMs has significantly improved over time; however, this comes at the cost of increased computational complexity, making them less accessible for resource-constrained applications such as mobile devices and assistive technologies. Alternatively, comparably smaller VLMs prioritize high-level scene descriptions, overlooking finer details that contribute to a richer understanding of an image. In this paper, we introduce a training-free framework that enhances caption diversity and informativeness by explicitly attending to distinct image regions using a comparably small VLM, BLIP, as the backbone. Our approach leverages structured segmentation to produce hierarchical representations that capture both global and localized semantics. Without requiring additional model training, we demonstrate that our method allows smaller VLMs to achieve performance comparable to larger models in terms of image-caption alignment, semantic integrity, and diversity. We evaluate our framework on MSCOCO, Flickr30k, and Nocaps test datasets, achieving a Div-2 score of 0.735, 0.750, and 0.748 for each dataset, respectively, while maintaining strong image-caption relevancy and semantic integrity with the human-annotated captions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
