使用视觉语言模型的知识丰富图像描述的束引导知识重放
Beam-Guided Knowledge Replay for Knowledge-Rich Image Captioning using Vision-Language Model
AI总结：
提出KRCapVLM框架，通过束搜索解码、注意力模块和训练调度器，提升视觉语言模型生成知识丰富图像描述的质量和泛化能力。
摘要：
生成信息丰富且知识丰富的图像描述对许多现有的描述模型仍然是一个挑战，这些模型通常生成缺乏具体性和上下文深度的通用描述。为了解决这一限制，我们提出了KRCapVLM，这是一个基于知识重放的、使用视觉语言模型的新型图像描述框架。我们结合束搜索解码以生成更多样和连贯的描述。我们还将基于注意力的模块集成到图像编码器中以增强特征表示。最后，我们采用训练调度器来提高稳定性并确保训练期间更平滑的收敛。这些提案在描述质量和知识识别方面带来了显著的提升。我们提出的模型在知识识别的准确性和生成描述的整体质量上都显示出明显的改进。它表现出更强的泛化到先前未见知识概念的能力，产生更具信息性和上下文相关性的描述。这些结果表明了我们的方法在增强模型在各种场景下生成有意义的、知识基础的描述能力方面的有效性。
英文摘要：
Generating informative and knowledge-rich image captions remains a challenge for many existing captioning models, which often produce generic descriptions that lack specificity and contextual depth. To address this limitation, we propose KRCapVLM, a knowledge replay-based novel image captioning framework using vision-language model. We incorporate beam search decoding to generate more diverse and coherent captions. We also integrate attention-based modules into the image encoder to enhance feature representation. Finally, we employ training schedulers to improve stability and ensure smoother convergence during training. These proposals accelerate substantial gains in both caption quality and knowledge recognition. Our proposed model demonstrates clear improvements in both the accuracy of knowledge recognition and the overall quality of generated captions. It shows a stronger ability to generalize to previously unseen knowledge concepts, producing more informative and contextually relevant descriptions. These results indicate the effectiveness of our approach in enhancing the model's capacity to generate meaningful, knowledge-grounded captions across a range of scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
