利用视觉价值模型扩展推理时搜索以提升视觉理解能力
Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehension
AI总结：
本文提出视觉价值模型VisVM，通过评估当前及未来句子质量的长期价值来引导VLM推理时搜索，减少幻觉并提升视觉描述质量，自训练后还可提升多模态基准性能。
摘要：
尽管视觉语言模型（VLMs）取得了显著进展，但仍缺乏通过扩展推理时计算来提升响应质量的有效方法。在近期大语言模型研究中，这一能力被认为是实现自改进模型的核心步骤。本文提出了视觉价值模型（VisVM），该模型能够引导VLM的推理时搜索，从而生成具有更好视觉理解能力的响应。具体而言，VisVM不仅评估当前搜索步骤中已生成句子的质量，还预测当前步骤可能产生的后续句子的质量，从而提供长期价值。通过这种方式，VisVM引导VLM避免生成容易产生幻觉或细节不足的句子，进而产出更高质量的响应。实验结果表明，与贪心解码以及使用其他视觉奖励信号的搜索方法相比，VisVM引导的搜索显著增强了VLM生成具有更丰富视觉细节和更少幻觉的描述性字幕的能力。此外，我们发现利用VisVM引导生成的字幕对模型进行自训练，可以提升VLM在多种多模态基准上的性能，这表明了开发自改进VLM的潜力。我们的价值模型和代码可在 https://github.com/si0wang/VisVM 获取。
英文摘要：
Despite significant advancements in vision-language models (VLMs), there lacks effective approaches to enhance response quality by scaling inference-time computation. This capability is known to be a core step towards the self-improving models in recent large language model studies. In this paper, we present Vision Value Model (VisVM) that can guide VLM inference-time search to generate responses with better visual comprehension. Specifically, VisVM not only evaluates the generated sentence quality in the current search step, but also anticipates the quality of subsequent sentences that may result from the current step, thus providing a long-term value. In this way, VisVM steers VLMs away from generating sentences prone to hallucinations or insufficient detail, thereby producing higher quality responses. Experimental results demonstrate that VisVM-guided search significantly enhances VLMs' ability to generate descriptive captions with richer visual details and fewer hallucinations, compared with greedy decoding and search methods with other visual reward signals. Furthermore, we find that self-training the model with the VisVM-guided captions improve VLM's performance across a wide range of multimodal benchmarks, indicating the potential for developing self-improving VLMs. Our value model and code are available at https://github.com/si0wang/VisVM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
