ViFP：一种用于增强视觉语言模型推理可靠性的视觉假阳性检测框架
ViFP: A Framework for Visual False Positive Detection to Enhance Reasoning Reliability in VLMs
AI总结：
本文提出ViFP框架，通过多轮问答构建推理路径、检测并纠正视觉语言模型中的假阳性推理，并引入VoC指标评估推理可靠性，在多个数据集上显著提升准确率并减少假阳性。
摘要：
在视觉语言模型（VLMs）的推理过程中，当模型产生正确答案但遵循错误的推理路径时，就会发生假阳性（FP）推理，从而削弱推理可靠性。现有方法主要依赖提示工程、知识蒸馏或强化学习来提高推理可靠性，这些方法都需要大量高质量数据，因而限制了实际适用性。很少有方法专注于直接检测和纠正假阳性。为了解决这些问题，我们提出了ViFP，一个用于视觉假阳性检测以增强视觉语言模型推理可靠性的框架。ViFP通过多轮问答构建有效的推理路径，并动态分析推理路径的一致性以识别潜在的假阳性。它还引入了一种针对性的推理链纠正机制来修改假阳性推理，从而提高逻辑一致性和准确性。最后，我们引入了一个可靠性评估指标VoC，该指标综合了答案准确率和假阳性率，为评估视觉语言模型是否不仅回答正确而且推理可靠提供了一个定量工具。我们在闭源视觉语言模型上的实验表明，ViFP在三个数据集上持续提升性能：A-OKVQA、OK-VQA和FVQA。在A-OKVQA上，ViFP将准确率提高了最多5.4%，比先前的最先进方法高出4.3%，并显著减少了假阳性的数量，验证了其在增强推理可靠性方面的优势。
英文摘要：
During reasoning in vision-language models (VLMs), false positive (FP) reasoning occurs when a model produces the correct answer but follows an incorrect reasoning path, resulting in undermined reasoning reliability. Existing approaches mainly rely on prompt engineering, knowledge distillation or reinforcement learning to improve reasoning reliability, both of which require large amounts of high-quality data and thus limit practical applicability. Few approaches have focused on directly detecting and correcting FPs. To address these issues, we propose ViFP, a framework for Visual False Positive Detection to Enhance Reasoning Reliability in VLMs. ViFP builds effective reasoning paths through multi-turn QA and dynamically analyzes the consistency of the reasoning path to identify potential FPs. It also introduces a targeted reasoning chain correction mechanism to modify FP reasoning, thereby improving logical consistency and accuracy. Finally, we introduce a reliability evaluation metric, VoC, which integrates answer accuracy and the FP rate, providing a quantitative tool to assess whether a VLM not only answers correctly but also reasons reliably. Our experiments on closed-source VLMs show that ViFP consistently improves performance across three datasets: A-OKVQA, OK-VQA, and FVQA. On A-OKVQA, ViFP improves accuracy by up to 5.4%, surpassing the previous state-of-the-art by 4.3%, and significantly reduces the number of FPs, validating its benefits in enhancing reasoning reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
