arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04413cs.CLcs.AIcs.MM
收藏

基于历史的迭代视觉推理与自我校正

History-Guided Iterative Visual Reasoning with Self-Correction

Nanjing University of Aeronautics and Astronautics(南京航空航天大学)

Xinglong Yang, Zhilin Peng, Zhanzhan Liu, Haochen Shi, Sheng-Jun Huang

AI总结:

H-GIVR框架通过迭代视觉推理与自我校正,显著提升多模态推理准确性并保持低计算成本。

摘要:

自一致性方法是提高多模态大语言模型(MLLMs)推理可靠性的核心技术。通过重复采样生成多个推理结果并通过投票选择最佳答案,它们在跨模态任务中发挥着重要作用。然而,大多数现有的自一致性方法局限于固定的``重复采样和投票''范式,并不重用历史推理信息。因此,模型难以主动纠正视觉理解错误并动态调整推理过程。受人类推理行为中重复验证和动态错误校正的启发,我们提出了H-GIVR框架。在迭代推理过程中,MLLM多次观察图像,并利用之前生成的答案作为后续步骤的参考,从而实现对错误的动态校正并提高答案的准确性。我们对五个数据集和三个模型进行了全面实验。结果表明,H-GIVR框架在保持低计算成本的同时,能显著提高跨模态推理的准确性。例如,在使用\texttt{Llama3.2-vision:11b}在ScienceQA数据集上时,模型需要每个问题平均2.57次响应才能达到78.90\%的准确率,比基线提高了107\%。

英文摘要:

Self-consistency methods are the core technique for improving the reasoning reliability of multimodal large language models (MLLMs). By generating multiple reasoning results through repeated sampling and selecting the best answer via voting, they play an important role in cross-modal tasks. However, most existing self-consistency methods are limited to a fixed ``repeated sampling and voting'' paradigm and do not reuse historical reasoning information. As a result, models struggle to actively correct visual understanding errors and dynamically adjust their reasoning during iteration. Inspired by the human reasoning behavior of repeated verification and dynamic error correction, we propose the H-GIVR framework. During iterative reasoning, the MLLM observes the image multiple times and uses previously generated answers as references for subsequent steps, enabling dynamic correction of errors and improving answer accuracy. We conduct comprehensive experiments on five datasets and three models. The results show that the H-GIVR framework can significantly improve cross-modal reasoning accuracy while maintaining low computational cost. For instance, using \texttt{Llama3.2-vision:11b} on the ScienceQA dataset, the model requires an average of 2.57 responses per question to achieve an accuracy of 78.90\%, representing a 107\% improvement over the baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑