BEiTScore: 一种基于高效交叉编码器的无参考图像描述评估方法
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
AI总结：
本文提出了一种无参考图像描述评估方法BEiTScore，通过高效的交叉编码器模型解决传统评估方法在计算成本和敏感性方面的不足，提出了一种新的评估指标，并在多种场景下验证了其优越的性能。
摘要：
图像描述评估仍是一个重大挑战，因为视觉-语言模型朝着生成长形式和上下文丰富的描述等更具挑战性的能力发展。最先进的评估度量标准涉及使用大型语言模型（LLMs）作为评判者的大量计算成本，或者受到标准CLIP基于编码器的限制，例如严格的令牌限制、缺乏细粒度敏感性或缺乏组合泛化能力，因为将描述视为“词袋”。我们提出了一种新的学习度量标准，以解决上述挑战，基于一个轻量级交叉编码器，其初始化来自视觉问答模型检查点，平衡了强大的权重初始化与计算效率。我们的训练方案使用精心编排的数据混合进行监督学习，特征是对抗性的LLM基于数据增强，以增强模型对细粒度视觉-语言错误的敏感性。我们还引入了一个新的基准，用于在多种场景中评估详细的描述评估。实验结果表明，所提出的度量标准在保持大规模基准测试、质量感知解码或奖励指导所需的效率的同时，实现了最先进的性能。
英文摘要：
Image captioning evaluation remains a significant challenge, as vision-language models evolve toward more challenging capabilities such as generating long-form and context-rich descriptions. State-of-the-art evaluation metrics involve extensive computational costs associated with the use of Large Language Models (LLMs) as judges, or instead suffer from the limitations of standard CLIP-based encoders, such as strict token limits, lack of fine-grained sensitivity, or lack of compositional generalization by treating captions as ``bags-of-words.'' We propose a new learned metric that tackles the aforementioned challenges, based on a lightweight cross-encoder that is initialized from a visual question-answering model checkpoint, balancing a strong weight initialization with computational efficiency. Our training scheme uses a carefully assembled data mixture for supervised learning, featuring adversarial LLM-based data augmentations to enhance model sensitivity to fine-grained visual-linguistic errors. We also introduce a new benchmark designed to assess detailed captioning evaluation across diverse scenarios. Experimental results demonstrate that the proposed metric achieves state-of-the-art performance while maintaining the efficiency required for large-scale benchmarking, quality-aware decoding, or reward guidance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
