DualCap：通过相似场景视觉提示的双检索增强轻量级图像描述
DualCap: Enhancing Lightweight Image Captioning via Dual Retrieval with Similar Scenes Visual Prompts
AI总结：
针对轻量级检索增强图像描述模型仅用文本提示导致视觉特征未增强、存在语义鸿沟的问题，提出DualCap双检索方法，结合文图与图图检索生成视觉提示并融合特征，在更少参数下实现了有竞争力的描述性能。
摘要：
近期的轻量级检索增强图像描述模型通常仅将检索到的数据用作文本提示，由此因未增强原始视觉特征而产生语义鸿沟，在处理物体细节或复杂场景时尤为明显。为解决这一局限，我们提出DualCap，这是一种通过从检索到的相似图像生成视觉提示来丰富视觉表征的新方法。我们的模型采用双检索机制：使用标准的图像到文本检索获取文本提示，同时采用新颖的图像到图像检索获取视觉相似的场景。具体而言，我们从视觉相似场景的描述中提取显著关键词和短语，以捕捉关键物体和相似细节。随后将这些文本特征编码，并通过一个轻量级的可训练特征融合网络与原始图像特征整合。大量实验表明，与此前的视觉提示描述方法相比，我们的方法在取得具有竞争力的性能的同时，所需的可训练参数更少。
英文摘要：
Recent lightweight retrieval-augmented image caption models often utilize retrieved data solely as text prompts, thereby creating a semantic gap by leaving the original visual features unenhanced, particularly for object details or complex scenes. To address this limitation, we propose $DualCap$, a novel approach that enriches the visual representation by generating a visual prompt from retrieved similar images. Our model employs a dual retrieval mechanism, using standard image-to-text retrieval for text prompts and a novel image-to-image retrieval to source visually analogous scenes. Specifically, salient keywords and phrases are derived from the captions of visually similar scenes to capture key objects and similar details. These textual features are then encoded and integrated with the original image features through a lightweight, trainable feature fusion network. Extensive experiments demonstrate that our method achieves competitive performance while requiring fewer trainable parameters compared to previous visual-prompting captioning approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
