双阶段价值引导推理与基于边际的奖励调整，用于快速且忠实的VLM描述生成
Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning
AI总结：
本文提出ViMaR，一种两阶段价值引导推理框架，通过边际奖励调整提升VLM描述的效率与保真度，实现4倍加速并跨模型泛化，自训练可提升视觉理解性能。
摘要：
尽管视觉语言模型（VLM）在推理时搜索方面取得了显著进展，但现有方法仍然计算成本高昂，并且容易产生未受惩罚的低置信度生成，这常常导致持续的幻觉。我们引入了 \textbf{基于边际奖励的价值引导推理（ViMaR）}，这是一个两阶段推理框架，通过结合时间差分价值模型与边际感知奖励调整，提高了效率和输出保真度。在第一阶段，我们执行单次遍历，从多样化的候选中识别出最高价值的描述。在第二阶段，我们选择性地仅细化那些被忽视或视觉基础薄弱的片段，从而消除频繁奖励的评估。经过校准的边际惩罚会抑制低置信度的延续，同时保留描述的丰富性。在多种VLM架构上的大量实验表明，ViMaR生成的描述显著更可靠、事实更准确、更详细且更具解释性，同时与现有价值引导方法相比实现了超过4倍的加速。具体而言，我们展示了仅基于LLaVA Mistral-7B训练的ViMaR，\textit{能有效泛化以引导更强的未见模型的解码}。为进一步验证这一点，我们将ViMaR适配到LLaVA-OneVision-Qwen2-7B中以引导生成，带来了描述质量的持续提升，并展示了稳健的跨模型引导能力。这种跨模型泛化凸显了ViMaR的灵活性和模块化，使其成为一种可扩展且可迁移的推理时解码策略。此外，当使用ViMaR生成的描述进行自训练时，底层模型在广泛的视觉理解基准上取得了显著提升，这凸显了快速、准确且自我改进的VLM流程的潜力。
英文摘要：
Despite significant advances in inference-time search for vision-language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallucinations. We introduce \textbf{Value-guided Inference with Margin-based Reward (ViMaR)}, a two-stage inference framework that improves both efficiency and output fidelity by combining a temporal-difference value model with a margin-aware reward adjustment. In the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive richness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$\times$ speedup compared to existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B, \textit{generalizes effectively to guide decoding in a stronger unseen model}. To further validate this, we adapt the ViMaR to steer generation in LLaVA-OneVision-Qwen2-7B, leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross-model generalization highlights ViMaR's flexibility and modularity, positioning it as a scalable and transferable inference-time decoding strategy. Furthermore, when ViMaR-generated captions are used for self-training, the underlying models achieve substantial gains across a broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
