ClaimDiff-RL: 通过视觉声明比较进行细粒度描述强化学习
ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison
AI总结：
提出ClaimDiff-RL框架，利用原子声明差异作为奖励单元，通过多模态判断器枚举视觉差异并分配错误类型和严重程度，以解决长描述强化学习中事实性与覆盖度的权衡问题。
摘要：
长格式图像描述揭示了强化学习中的奖励粒度问题：描述被整体判断，而重要错误发生在单个视觉声明层面。一个好的密集描述应既忠实又信息丰富，避免幻觉而不遗漏显著细节。然而，成对偏好、基于参考的指标和整体标量奖励将这些局部错误压缩为单个序列级信号，模糊了事实性与覆盖度之间的权衡。我们引入ClaimDiff-RL框架，该框架使用基于参考的原子声明差异作为描述强化学习的奖励单元。给定一张图像、一个演员描述和一个参考描述，多模态判断器枚举视觉上可区分的差异，针对图像验证每个差异，分配开放词汇的错误类型和严重程度，并生成每个差异的统计信息用于奖励组合。这使得幻觉声明和遗漏的显著事实可以分别测量和调整。实验表明，整体标量奖励可以通过增加遗漏事实来减少幻觉，而ClaimDiff-RL揭示了这种忠实性与覆盖度的权衡，并实现了更平衡的操作点。在包含160张图像的人工标注诊断基准、公开描述基准和VQA基准上，ClaimDiff-RL改善了幻觉-遗漏事实平衡，保留了通用能力，甚至在多个细粒度能力维度（如物体计数、空间关系和场景识别）上超越了Gemini-3-Pro-Preview。这些结果表明，类型化、可验证的声明差异是细粒度且可诊断的描述强化学习的有效奖励单元。
英文摘要：
Long-form image captioning exposes a reward granularity problem in RL: captions are judged as whole sequences, while the important errors occur at the level of individual visual claims. A good dense caption should be both faithful and informative, avoiding hallucination without omitting salient details. Yet pairwise preferences, reference-based metrics, and holistic scalar rewards compress these local errors into a single sequence-level signal, obscuring the tradeoff between factuality and coverage. We introduce ClaimDiff-RL, a framework that uses reference-conditioned atomic claim differences as the reward unit for caption RL. Given an image, an actor caption, and a reference caption, a multimodal judge enumerates visually grounded differences, verifies each difference against the image, assigns open-vocabulary error types and severity levels, and produces per-difference statistics for reward composition. This makes hallucinated claims and omitted salient facts separately measurable and tunable. Experiments show that holistic scalar rewards can reduce hallucination by increasing missing facts, while ClaimDiff-RL exposes this faithfulness and coverage tradeoff and enables more balanced operating points. On a 160-image human-labeled diagnostic benchmark, public captioning benchmarks, and VQA benchmarks, ClaimDiff-RL improves the hallucination--missing-fact balance, preserves general capability, and even surpasses Gemini-3-Pro-Preview on several fine-grained Capability dimensions such as object counting, spatial relations, and scene recognition. These results suggest that typed, verifiable claim differences are an effective reward unit for fine-grained and diagnosable caption RL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
