ViCrit：面向视觉语言模型视觉感知的可验证强化学习代理任务
ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
AI总结：
针对视觉语言模型视觉感知强化学习缺乏兼具挑战性与可验证性任务的问题，提出ViCrit代理任务与ViCrit-Bench基准，通过定位字幕细微幻觉训练模型，有效提升多场景视觉感知能力与泛化性。
摘要：
强化学习（RL）在使用数学推理、代码生成等兼具挑战性与易验证性的任务微调大语言模型（LLM）方面已展现出显著成效。然而，由于同时具备挑战性与明确可验证性的视觉中心任务稀缺，将这一成功拓展至视觉语言模型（VLM）的视觉感知领域一直受阻。为此，我们提出ViCrit（Visual Caption Hallucination Critic，视觉字幕幻觉评判器），这是一种强化学习代理任务，用于训练VLM定位注入到人工撰写图像字幕段落中的细微合成视觉幻觉。我们以200词的字幕为基础，注入单个细微的视觉描述错误——修改关于物体、属性、数量或空间关系的少量词汇——并要求模型结合图像与修改后的字幕，定位出被篡改的文本片段。该任务设定保留了完整的感知难度，同时提供了易于计算、结果明确的二值精确匹配奖励。经ViCrit任务训练的模型在多种VLM基准测试中均取得大幅提升。至关重要的是，这些性能提升可迁移至自然图像训练数据之外的抽象图像推理与视觉数学任务，展现出模型是在学习感知而非仅记忆见过的物体的潜力。为便于评估，我们进一步推出ViCrit-Bench，这是一个类别均衡的诊断基准，可系统地探测不同图像领域和错误类型下的感知错误。综上，我们的研究结果表明，细粒度幻觉评判是增强VLM视觉感知的一种有效且可泛化的优化目标。
英文摘要：
Reinforcement learning (RL) has shown great effectiveness for fine-tuning large language models (LLMs) using tasks that are challenging yet easily verifiable, such as math reasoning or code generation. However, extending this success to visual perception in vision-language models (VLMs) has been impeded by the scarcity of vision-centric tasks that are simultaneously challenging and unambiguously verifiable. To this end, we introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions. Starting from a 200-word captions, we inject a single, subtle visual description error-altering a few words on objects, attributes, counts, or spatial relations-and task the model to pinpoint the corrupted span given the image and the modified caption. This formulation preserves the full perceptual difficulty while providing a binary, exact-match reward that is easy to compute and unambiguous. Models trained with the ViCrit Task exhibit substantial gains across a variety of VL benchmarks. Crucially, the improvements transfer beyond natural-image training data to abstract image reasoning and visual math, showing promises of learning to perceive rather than barely memorizing seen objects. To facilitate evaluation, we further introduce ViCrit-Bench, a category-balanced diagnostic benchmark that systematically probes perception errors across diverse image domains and error types. Together, our results demonstrate that fine-grained hallucination criticism is an effective and generalizable objective for enhancing visual perception in VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
