DENEB：一种面向图像描述生成、对幻觉具有鲁棒性的自动评估指标
DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning
AI总结：
本文提出抗幻觉的图像描述自动评估指标 DENEB，利用 Sim-Vec Transformer 同时建模图像、候选描述与多参考描述，并基于 Nebula 人工标注数据集训练，在多个基准上达到无 LLM 指标的最优水平。
摘要：
在本工作中，我们应对为图像描述生成开发自动评估指标的挑战，尤其关注对幻觉的鲁棒性。现有指标往往不足以处理幻觉，主要原因是它们将候选描述与多方面参考描述进行比较的能力有限。为弥补这一缺陷，我们提出 DENEB，这是一种专门针对幻觉具有鲁棒性的新型有监督自动评估指标。DENEB 融合了 Sim-Vec Transformer，该机制可同时处理多个参考描述，从而有效捕捉图像、候选描述与参考描述之间的相似性。为训练 DENEB，我们构建了多样且均衡的 Nebula 数据集，包含 32,978 张图像，并配有由 805 名标注者提供的人工判断。我们证明，DENEB 在 FOIL、Composite、Flickr8K-Expert、Flickr8K-CF、Nebula 和 PASCAL-50S 数据集上的现有无 LLM 指标中取得了最先进性能，验证了其有效性及对幻觉的鲁棒性。
英文摘要：
In this work, we address the challenge of developing automatic evaluation metrics for image captioning, with a particular focus on robustness against hallucinations. Existing metrics are often inadequate for handling hallucinations, primarily due to their limited ability to compare candidate captions with multifaceted reference captions. To address this shortcoming, we propose DENEB, a novel supervised automatic evaluation metric specifically robust against hallucinations. DENEB incorporates the Sim-Vec Transformer, a mechanism that processes multiple references simultaneously, thereby efficiently capturing the similarity between an image, a candidate caption, and reference captions. To train DENEB, we construct the diverse and balanced Nebula dataset comprising 32,978 images, paired with human judgments provided by 805 annotators. We demonstrated that DENEB achieves state-of-the-art performance among existing LLM-free metrics on the FOIL, Composite, Flickr8K-Expert, Flickr8K-CF, Nebula, and PASCAL-50S datasets, validating its effectiveness and robustness against hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
