arXiv 2605.280232026-05-27cs.CVcs.AIcs.CLcs.MM

VCap: 用于弱到强视觉字幕的超几何奖励

VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning

Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳分校）；Chinese Academy of Sciences（中国科学院）；Kuaishou Technology（快手科技）

Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang, Yankai Yang, Yancheng Long, Yiyang Fan, Xuanyu Zheng, Haonan Fan, Kaiyu Jiang, Tianke Zhang, Changyi Liu, Bin Wen, Fan Ya… 展开作者

Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang, Yankai Yang, Yancheng Long, Yiyang Fan, Xuanyu Zheng, Haonan Fan, Kaiyu Jiang, Tianke Zhang, Changyi Liu, Bin Wen, Fan Yang, Tingting Gao, Han Li, Chun Yuan

备注：: 28 pages, 8 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出VCap，一种证人-裁判奖励机制，通过超几何分布级别的精度验证视觉信号中参考字幕与策略生成字幕之间的事实一致性，实现弱到强泛化，在多个图像和视频字幕基准上超越SOTA模型。

摘要：

视觉字幕要求模型忠实捕捉视觉内容，同时最小化遗漏和幻觉。作为字幕的主导范式，多模态大语言模型通过扩展和高质量数据取得了强大性能。最近，强化学习成为推动多模态大语言模型向更高精度和更广覆盖的关键途径，然而，现有字幕奖励设计未能提供细粒度且可靠的事实验证信号，限制了其有效性。为解决这一问题，我们提出VCap，一种证人-裁判奖励，将参考字幕（证人）与视觉信号（裁判）配对。通过明确验证基于视觉信号的参考字幕与策略生成字幕之间的事实一致性，VCap提供了具有超几何分布级别精度的奖励信号用于字幕质量验证。该设计即使在不完美的参考下也能实现有效学习，促进强化学习训练中的弱到强泛化。在我们的实验中，使用VCap训练的8B模型在多个图像和视频字幕基准上优于开源和闭源的最先进模型。人工评估进一步证实了其与事实正确性的强对齐。此外，VCap提升了多模态大语言模型的感知能力，跨任务泛化，并超越了最佳N蒸馏，挑战了先前关于强化学习与视觉推理的假设。

英文摘要：

Visual captioning requires models to capture visual content faithfully while minimizing both omission and hallucination. As the dominant paradigm for captioning, MLLMs have achieved strong performance through scaling and high-quality data. Recently, RL has emerged as a key route to driving MLLMs toward higher precision and broader coverage, however, existing reward designs for captioning fail to provide fine-grained and reliable signals for factual verification, limiting their effectiveness. To address this, we propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator). By explicitly verifying factual consistency between the reference and policy-generated captions grounded in the visual signal, VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification. This design enables effective learning even from imperfect references, facilitating weak-to-strong generalization in RL training. In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks. Human evaluation further confirms its strong alignment with factual correctness. Additionally, VCap improves MLLM perceptual capability, generalizes across tasks, and surpasses best-of-N distillation, challenging prior assumptions about RLVR.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。