EgoExoBench：多模态大语言模型中第一人称与第三人称视角视频理解基准
EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs
AI总结：
本文提出首个面向第一/第三人称视频理解与推理的基准 EgoExoBench，通过 7,300 多个问答对评估 13 个 MLLMs，揭示其跨视角语义、关联及时序推理短板。
摘要：
在第一人称（自我中心，egocentric）和第三人称（外部中心，exocentric）视角之间迁移并整合知识，是人类智能的内在能力，使人类能够向他人学习，并传递自身经验中的洞见。尽管多模态大语言模型（MLLMs）发展迅速，但其执行此类跨视角推理的能力仍未得到探索。为解决这一问题，我们提出 EgoExoBench，这是首个面向自我中心—外部中心视频理解与推理的基准。EgoExoBench 基于公开可用数据集构建，包含超过 7,300 个问答对，涵盖组织为三大核心挑战的 11 个子任务：语义对齐、视角关联和时序推理。我们评估了 13 个最先进的 MLLMs，发现尽管这些模型在单视角任务上表现出色，但在自我中心—外部中心情境中，它们难以跨视角对齐语义、准确关联不同视角并推断时序动态。我们希望 EgoExoBench 能为具身智能体和追求类人跨视角智能的智能助手研究提供宝贵资源。
英文摘要：
Transferring and integrating knowledge across first-person (egocentric) and third-person (exocentric) viewpoints is intrinsic to human intelligence, enabling humans to learn from others and convey insights from their own experiences. Despite rapid progress in multimodal large language models (MLLMs), their ability to perform such cross-view reasoning remains unexplored. To address this, we introduce EgoExoBench, the first benchmark for egocentric-exocentric video understanding and reasoning. Built from publicly available datasets, EgoExoBench comprises over 7,300 question-answer pairs spanning eleven sub-tasks organized into three core challenges: semantic alignment, viewpoint association, and temporal reasoning. We evaluate 13 state-of-the-art MLLMs and find that while these models excel on single-view tasks, they struggle to align semantics across perspectives, accurately associate views, and infer temporal dynamics in the ego-exo context. We hope EgoExoBench can serve as a valuable resource for research on embodied agents and intelligent assistants seeking human-like cross-view intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
