RACap：面向轻量级检索增强图像描述的关系感知提示方法
RACap: Relation-Aware Prompting for Lightweight Retrieval-Augmented Image Captioning
AI总结：
本文提出关系感知检索增强图像描述模型 RACap，通过挖掘检索描述中的结构化关系语义并识别图像异构对象，以仅 10.8M 可训练参数取得优于以往轻量级模型的性能。
摘要：
近年来，检索增强的图像描述方法引入外部知识，以弥补在理解复杂场景方面的不足。然而，现有方法在关系建模上仍面临挑战：（1）语义提示的表示粒度过粗，难以捕捉细粒度关系；（2）这些方法缺乏对图像对象及其语义关系的显式建模。为解决上述局限，我们提出 RACap，一种关系感知的检索增强图像描述模型，该模型不仅从检索到的描述文本中挖掘结构化关系语义，还能从图像中识别异构对象。RACap 有效检索包含异构视觉信息的结构化关系特征，从而增强语义一致性和关系表达能力。实验结果表明，RACap 仅使用 10.8M 可训练参数，即取得了优于以往轻量级图像描述模型的性能。
英文摘要：
Recent retrieval-augmented image captioning methods incorporate external knowledge to compensate for the limitations in comprehending complex scenes. However, current approaches face challenges in relation modeling: (1) the representation of semantic prompts is too coarse-grained to capture fine-grained relationships; (2) these methods lack explicit modeling of image objects and their semantic relationships. To address these limitations, we propose RACap, a relation-aware retrieval-augmented model for image captioning, which not only mines structured relation semantics from retrieval captions, but also identifies heterogeneous objects from the image. RACap effectively retrieves structured relation features that contain heterogeneous visual information to enhance the semantic consistency and relational expressiveness. Experimental results show that RACap, with only 10.8M trainable parameters, achieves superior performance compared to previous lightweight captioning models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
