EgoVITA：学习规划与验证以实现自我中心视频推理
EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning
AI总结：
提出EgoVITA框架，通过先规划自我中心动作序列再以第三方视角验证时空一致性的方法，提升自我中心视频推理的准确性。
摘要：
自我中心视频理解需要在部分可观测和不断变化的视角下进行程序性推理。当前的多模态大语言模型（MLLMs）在此场景中表现不佳，常常生成看似合理但视觉不一致或缺乏依据的响应。我们提出$\textbf{EgoVITA}$框架，将自我中心视频推理分解为结构化的$\textit{规划-验证}$过程。模型首先生成一个$\textbf{自我中心规划}$：从第一人称视角出发的预期动作因果序列。然后，通过一个$\textbf{外部中心验证}$阶段对该规划进行评估，该阶段使用对同一视频的第三人称推理来验证其时空和逻辑一致性，无需外部中心视频输入。这种分解使得无需配对的自我-外部中心监督即可实现跨视角反馈。为了训练这一推理过程，我们采用组相对策略优化（GRPO）和两种密集奖励信号：一种将预期动作与后续视觉观察联系起来，另一种强化一致的第三人称验证。$\textbf{EgoVITA}$在自我中心推理基准上取得了最先进的性能，在EgoBlind上比Qwen2.5-VL-7B高出$\mathbf{+7.7}$，在EgoOrient上高出$\mathbf{+4.4}$，同时在仅使用$52k$个训练样本的情况下，在外部中心视频任务上保持了强大的泛化能力。
英文摘要：
Egocentric video understanding requires procedural reasoning under partial observability and continuously shifting viewpoints. Current multimodal large language models (MLLMs) struggle with this setting, often generating plausible but visually inconsistent or weakly grounded responses. We introduce $\textbf{EgoVITA}$, a framework that decomposes egocentric video reasoning into a structured $\textit{plan-then-verify}$ process. The model first generates an $\textbf{egocentric plan}$: a causal sequence of anticipated actions from a first-person perspective. This plan is then evaluated by an $\textbf{exocentric verification}$ stage that uses third-person reasoning over the same video to verify its spatiotemporal and logical consistency, without exocentric video input. This decomposition enables cross-perspective feedback without requiring paired ego-exo supervision. To train this reasoning process, we adopt Group Relative Policy Optimization (GRPO) with two dense reward signals: one that grounds anticipated actions in subsequent visual observations and another that reinforces consistent third-person verification. $\textbf{EgoVITA}$ achieves state-of-the-art performance on egocentric reasoning benchmarks, outperforming Qwen2.5-VL-7B by $\mathbf{+7.7}$ on EgoBlind and $\mathbf{+4.4}$ on EgoOrient, while maintaining strong generalization on exocentric video tasks with only $52k$ training samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
