BRIDGE：通过更强的视觉线索弥合图像描述评估中的差距
BRIDGE: Bridging Gaps in Image Captioning Evaluation with Stronger Visual Cues
AI总结：
本文提出BRIDGE，一种可学习且无需参考的图像描述评估指标，通过将视觉特征映射为密集向量并整合到多模态伪描述中，有效编码细粒度细节并惩罚幻觉，在多个数据集上达到最先进水平。
摘要：
在评估机器生成的图像描述时，有效地与人类判断保持一致是一个复杂但极具吸引力的挑战。现有的评估指标（如 CIDEr 或 CLIP-Score）在这方面存在不足，因为它们没有考虑到对应的图像，或者缺乏编码细粒度细节和惩罚幻觉的能力。为了克服这些问题，本文提出了 BRIDGE，一种新的可学习且无需参考的图像描述评估指标。该指标采用一个新颖的模块将视觉特征映射为密集向量，并将其整合到在评估过程中构建的多模态伪描述中。这种方法产生了一个多模态指标，它能够适当地整合来自输入图像的信息，而无需依赖参考描述，从而弥合了人类判断与机器生成的图像描述之间的差距。跨越多个数据集的实验表明，与现有的无参考评估分数相比，我们的提案取得了最先进的结果。我们的源代码和训练模型已在以下网址公开提供：https://github.com/aimagelab/bridge-score。
英文摘要：
Effectively aligning with human judgment when evaluating machine-generated image captions represents a complex yet intriguing challenge. Existing evaluation metrics like CIDEr or CLIP-Score fall short in this regard as they do not take into account the corresponding image or lack the capability of encoding fine-grained details and penalizing hallucinations. To overcome these issues, in this paper, we propose BRIDGE, a new learnable and reference-free image captioning metric that employs a novel module to map visual features into dense vectors and integrates them into multi-modal pseudo-captions which are built during the evaluation process. This approach results in a multimodal metric that properly incorporates information from the input image without relying on reference captions, bridging the gap between human judgment and machine-generated image captions. Experiments spanning several datasets demonstrate that our proposal achieves state-of-the-art results compared to existing reference-free evaluation scores. Our source code and trained models are publicly available at: https://github.com/aimagelab/bridge-score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
