Rigel: 基于自蒸馏分数自适应的图像与视频字幕评估
Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation
AI总结：
提出Rigel评估指标，通过自蒸馏从冻结LLM提取评估专用评分头，解决大词汇语言模型与小标签集不匹配问题，在视频字幕评估中超越现有方法。
摘要：
图像和视频字幕的自动评估对于多模态系统的基准测试至关重要，尽管标准评估指标与人类判断的一致性有限。最近使用大语言模型（LLM）的方法，通常称为LLM-as-a-Judge，提高了与人类判断的一致性，但仍然存在大词汇语言建模与在小标签集上评估之间的不匹配。为了解决这个问题，我们提出了Rigel，一种基于自蒸馏分数自适应的图像和视频字幕自动评估指标。该指标采用从冻结LLM蒸馏出的评估专用评分头，在任务对齐的空间中捕获判断信号，而不依赖大词汇标记集。然后，我们用人类判断数据微调LLM主干。为了训练Rigel，我们构建了Vid-Lepus数据集，包含3,338个视频片段、33,380个参考字幕和5,637个候选字幕。在多个基准上的实验表明，Rigel优于最先进的指标，在无参考设置下在ActivityNet-Fact上实现了超过10个百分点的改进。
英文摘要：
Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited alignment with human judgments. Recent approaches using large language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved alignment with human judgments but still suffer from a mismatch between large-vocabulary language modeling and evaluation over a small label set. To address this, we propose Rigel, an automatic evaluation metric for image and video captioning, based on self-distilled score adaptation. The metric employs an evaluation-specific scoring head distilled from a frozen LLM, which captures judgment signals in a task-aligned space without relying on large-vocabulary token sets. We then refine the LLM backbone with human judgment data. To train Rigel, we constructed the Vid-Lepus dataset, which contains 3,338 video clips, 33,380 reference captions, and 5,637 candidate captions. Experiments on multiple benchmarks show that Rigel outperforms state-of-the-art metrics, achieving over 10-point improvements on ActivityNet-Fact in the reference-free setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
