多模态大语言模型中的忠实优先推理、规划与行动
Faithful-First Reasoning, Planning, and Acting for Multimodal LLMs
AI总结：
本文提出Faithful-First RPA框架，通过逐步监督提升多模态推理的忠实度，实验表明其在多个基准上提升了24%的感知忠实度，且不降低任务准确性。
摘要：
多模态大语言模型（MLLMs）常因推理链偏离视觉证据或与最终预测矛盾而缺乏忠实性。我们提出Faithful-First Reasoning, Planning, and Acting（RPA）框架，其中FaithEvi通过评估中间推理的忠实性提供逐步监督，FaithAct利用这些信号在推理过程中规划和执行忠实性感知的动作。在多个多模态推理基准上的实验表明，忠实优先RPA在提示基于和工具增强的推理框架上将感知忠实度提高了高达24%，且不降低任务准确性。我们的分析表明，将忠实性视为指导原则可以引导感知忠实的推理轨迹并减轻幻觉行为。本工作因此建立了评估和强制多模态推理中忠实性的统一框架。代码见https://github.com/lijunxian111/Faithful-First-RPA。
英文摘要：
Multimodal Large Language Models (MLLMs) frequently suffer from unfaithfulness, generating reasoning chains that drift from visual evidence or contradict final predictions. We propose Faithful-First Reasoning, Planning, and Acting (RPA) framework in which FaithEvi provides step-wise and chain-level supervision by evaluating the faithfulness of intermediate reasoning, and FaithAct uses these signals to plan and execute faithfulness-aware actions during inference. Experiments across multiple multimodal reasoning benchmarks show that faithful-first RPA improves perceptual faithfulness by up to 24% over prompt-based and tool-augmented reasoning frameworks, without degrading task accuracy. Our analysis shows that treating faithfulness as a guiding principle perceptually faithful reasoning trajectories and mitigates hallucination behavior. This work thereby establishes a unified framework for both evaluating and enforcing faithfulness in multimodal reasoning. Code is at https://github.com/lijunxian111/Faithful-First-RPA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
