iReasoner: 一种面向轨迹的内在推理监督方法，用于自演化的大多模态模型
iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models
AI总结：
本文提出iReasoner，一种自演化框架，通过显式引导推理链和奖励内部一致性来提升大模型的隐式推理能力，在无监督设置下实现了多模态推理基准的性能提升。
摘要：
最近的研究表明，大多模态模型（LMMs）可以通过自博弈和内在反馈从无标签数据中自我改进。然而，现有的自演化框架主要奖励最终结果，尽管中间推理对于视觉基础决策至关重要，但其约束较弱。我们提出iReasoner，一种自演化框架，通过显式引导推理链（CoT）并奖励其内部一致性来提升LMM的隐式推理能力。在无标签图像上进行Proposer--Solver循环时，iReasoner通过在中间推理步骤上定义轨迹感知信号，增强结果层面的内在奖励，提供无需真实标签或外部评判的学习信号，以区分导向相同答案的不同推理路径。从Qwen2.5-VL-7B开始，iReasoner在完全无监督的后训练阶段，在多样化的多模态推理基准上实现了最高+2.1分的提升。我们希望这项工作能成为在纯无监督设置中实现推理感知自改进的LMMs的起点。我们的代码可在https://meghanaasunil.github.io/iReasoner上获取。
英文摘要：
Recent work shows that large multimodal models (LMMs) can self-improve from unlabeled data via self-play and intrinsic feedback. Yet existing self-evolving frameworks mainly reward final outcomes, leaving intermediate reasoning weakly constrained despite its importance for visually grounded decision making. We propose iReasoner, a self-evolving framework that improves an LMM's implicit reasoning by explicitly eliciting chain-of-thought (CoT) and rewarding its internal agreement. In a Proposer--Solver loop over unlabeled images, iReasoner augments outcome-level intrinsic rewards with a trajectory-aware signal defined over intermediate reasoning steps, providing learning signals that distinguish reasoning paths leading to the same answer without ground-truth labels or external judges. Starting from Qwen2.5-VL-7B, iReasoner yields up to $+2.1$ points across diverse multimodal reasoning benchmarks under fully unsupervised post-training. We hope this work serves as a starting point for reasoning-aware self-improvement in LMMs in purely unsupervised settings. Our code is available at https://meghanaasunil.github.io/iReasoner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
