RubiCap：基于评分标准的强化学习用于密集图像描述
RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning
AI总结：
RubiCap通过基于评分标准的强化学习方法，在密集图像描述任务中实现了更高的胜率和词效，优于监督蒸馏和传统RL方法。
摘要：
密集图像描述对于视觉-语言预训练和文本到图像生成中的跨模态对齐至关重要，但扩展专家级注释是极其昂贵的。尽管通过强大视觉-语言模型（VLMs）进行合成描述是一个实用的替代方案，但监督蒸馏通常会产生输出多样性有限和泛化能力弱的问题。强化学习（RL）可以克服这些限制，但其成功主要集中在可验证的领域，这些领域依赖于确定性检查器——在开放式的描述生成中这种资源并不可用。我们通过RubiCap，一种新的RL框架，从LLM生成的评分标准中提取细粒度、样本特定的奖励信号来解决这一瓶颈。RubiCap首先组建一个多样化的候选描述委员会，然后使用LLM评分标准生成器提取当前策略的一致优势并诊断缺陷。这些见解被转换为明确的评估标准，使LLM判官能够分解整体质量评估并用结构化的、多方面的评估取代粗略的标量奖励。在广泛的基准测试中，RubiCap在CapArena上实现了最高的胜率，优于监督蒸馏、先前的RL方法、人类专家注释和GPT-4V增强的输出。在CaptionQA上，它展示了优越的词效：我们的7B模型可与Qwen2.5-VL-32B-Instruct相媲美，而我们的3B模型超越了其7B版本。值得注意的是，使用紧凑的RubiCap-3B作为描述生成器可以产生比那些在专有模型描述上训练的VLM更强的预训练VLM。
英文摘要：
Dense image captioning is critical for vision-language pretraining and for text-to-image generation, but scaling expert-quality annotations is prohibitively expensive. While synthesizing captions from strong vision-language models (VLMs) is a practical alternative, supervised distillation often yields limited output diversity and weak generalization. Reinforcement learning (RL) could overcome these, but its successes have so far been concentrated in verifiable domains that rely on deterministic checkers---a luxury not available in open-ended captioning. We address this verification bottleneck with RubiCap, an RL framework that derives fine-grained, sample-specific rewards from LLM-written rubrics. For each image, an LLM rubric writer compares captions from a diverse committee of VLMs to identify consensus strengths and diagnose the current policy's deficiencies. These findings are converted into explicit evaluation criteria, enabling an LLM judge to decompose quality assessment and replace coarse scalar rewards with structured, multi-faceted assessments. Across extensive benchmarks, RubiCap achieves the highest win rates on CapArena, outperforming supervised distillation, prior RL methods, human-expert annotations, and GPT-4V-augmented outputs. On CaptionQA, it shows superior word efficiency: our 7B model matches Qwen2.5-VL-32B-Instruct, and our 3B model surpasses its 7B counterpart. Remarkably, using a compact RubiCap-3B as a captioner produces stronger pretrained VLMs than those trained on captions from proprietary models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
