BalCapRL：一种基于RL的MLLM图像描述的平衡框架
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
AI总结：
本文提出BalCapRL框架，通过联合优化正确性、参考覆盖和语言质量，解决图像描述中因评价指标狭窄导致的权衡问题，并通过改进的奖励解耦归一化和长度条件奖励遮蔽提升性能。
摘要：
图像描述是计算机视觉中最基本的任务之一。由于其开放性，它在多模态大语言模型（MLLMs）时代受到了广泛关注。为获得更详细准确的描述，最近的研究越来越多地转向强化学习（RL）。然而，现有的描述RL方法和评估指标往往强调狭窄的描述质量概念，导致在描述核心维度上产生权衡。例如，以实用性为导向的目标可以鼓励噪声、幻觉或过长的描述，从而提高下游问答性能，但损害流畅性；而竞技场式目标则倾向于选择流畅但通用的描述，具有有限的实用性。为此，我们提出了一种更平衡的RL框架，联合优化实用性感知的正确性、参考覆盖和语言质量。为了有效优化由此产生的连续多目标奖励公式，我们应用GDPO-style奖励解耦归一化来处理连续值的描述奖励，并证明其在Vanilla GRPO上表现更优。此外，我们引入了长度条件奖励遮蔽，以获得更合适的描述长度惩罚。在LLaVA-1.5-7B和Qwen2.5-VL 3B和7B基模型上，我们的方法一致提升了描述质量，不同模型的峰值提升分别为+13.6 DCScore、+9.0 CaptionQA和+29.0 CapArena。
英文摘要：
Image captioning is one of the most fundamental tasks in computer vision. Owing to its open-ended nature, it has received significant attention in the era of multimodal large language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions that improve downstream question answering while harming fluency, whereas arena-style objectives can favor fluent but generic descriptions with limited usefulness. To address this, we propose a more balanced RL framework that jointly optimizes utility-aware correctness, reference coverage, and linguistic quality. In order to effectively optimize the resulting continuous multi-objective reward formulation, we apply GDPO-style reward-decoupled normalization to continuous-valued captioning rewards and show that it improves performance over vanilla GRPO. Additionally, we introduce length-conditional reward masking, yielding a more suitable length penalty for captioning. Across LLaVA-1.5-7B and Qwen2.5-VL 3B and 7B base models, our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 CapArena across different models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
